In this lecture notebook we'll be looking at an introduction to Grad-CAM, a powerful technique for interpreting Convolutional Neural Networks. Grad-CAM stands for Gradient-weighted Class Activation Mapping.
CNN's are very flexible models and their great predictive power comes at the cost of losing interpretability (something that is true for all Artificial Neural Networks). Grad-CAM attempts to solve this by giving us a graphical visualisation of parts of an image that are the most relevant for the CNN when predicting a particular class.
Aside from working on some Grad-CAM concepts we'll also look at how we can use Keras to access some concrete information of our model. Let's dive into it!
import keras
from keras import backend as K
from util import *
Using TensorFlow backend.
The load_C3M3_model()
function has been taken care of and its internals are out of the scope of this notebook. But if it intrigues you, you can take a look at it in util.py
# Load the model we are going to be using
model = load_C3M3_model()
Got loss weights Loaded DenseNet Added layers Compiled Model Loaded Weights
As you may already know, we can check the architecture of our model using the summary()
method.
After running the code block below we’ll see that this model has a lot of layers. One advantage of Grad-CAM over previous attempts of interpreting CNN's (such as CAM) is that it is architecture agnostic. This means it can be used for CNN's with complex architectures such as this one:
# Print all of the model's layers
model.summary()
__________________________________________________________________________________________________ Layer (type) Output Shape Param # Connected to ================================================================================================== input_1 (InputLayer) (None, None, None, 3 0 __________________________________________________________________________________________________ zero_padding2d_1 (ZeroPadding2D (None, None, None, 3 0 input_1[0][0] __________________________________________________________________________________________________ conv1/conv (Conv2D) (None, None, None, 6 9408 zero_padding2d_1[0][0] __________________________________________________________________________________________________ conv1/bn (BatchNormalization) (None, None, None, 6 256 conv1/conv[0][0] __________________________________________________________________________________________________ conv1/relu (Activation) (None, None, None, 6 0 conv1/bn[0][0] __________________________________________________________________________________________________ zero_padding2d_2 (ZeroPadding2D (None, None, None, 6 0 conv1/relu[0][0] __________________________________________________________________________________________________ pool1 (MaxPooling2D) (None, None, None, 6 0 zero_padding2d_2[0][0] __________________________________________________________________________________________________ conv2_block1_0_bn (BatchNormali (None, None, None, 6 256 pool1[0][0] __________________________________________________________________________________________________ conv2_block1_0_relu (Activation (None, None, None, 6 0 conv2_block1_0_bn[0][0] __________________________________________________________________________________________________ conv2_block1_1_conv (Conv2D) (None, None, None, 1 8192 conv2_block1_0_relu[0][0] __________________________________________________________________________________________________ conv2_block1_1_bn (BatchNormali (None, None, None, 1 512 conv2_block1_1_conv[0][0] __________________________________________________________________________________________________ conv2_block1_1_relu (Activation (None, None, None, 1 0 conv2_block1_1_bn[0][0] __________________________________________________________________________________________________ conv2_block1_2_conv (Conv2D) (None, None, None, 3 36864 conv2_block1_1_relu[0][0] __________________________________________________________________________________________________ conv2_block1_concat (Concatenat (None, None, None, 9 0 pool1[0][0] conv2_block1_2_conv[0][0] __________________________________________________________________________________________________ conv2_block2_0_bn (BatchNormali (None, None, None, 9 384 conv2_block1_concat[0][0] __________________________________________________________________________________________________ conv2_block2_0_relu (Activation (None, None, None, 9 0 conv2_block2_0_bn[0][0] __________________________________________________________________________________________________ conv2_block2_1_conv (Conv2D) (None, None, None, 1 12288 conv2_block2_0_relu[0][0] __________________________________________________________________________________________________ conv2_block2_1_bn (BatchNormali (None, None, None, 1 512 conv2_block2_1_conv[0][0] __________________________________________________________________________________________________ conv2_block2_1_relu (Activation (None, None, None, 1 0 conv2_block2_1_bn[0][0] __________________________________________________________________________________________________ conv2_block2_2_conv (Conv2D) (None, None, None, 3 36864 conv2_block2_1_relu[0][0] __________________________________________________________________________________________________ conv2_block2_concat (Concatenat (None, None, None, 1 0 conv2_block1_concat[0][0] conv2_block2_2_conv[0][0] __________________________________________________________________________________________________ conv2_block3_0_bn (BatchNormali (None, None, None, 1 512 conv2_block2_concat[0][0] __________________________________________________________________________________________________ conv2_block3_0_relu (Activation (None, None, None, 1 0 conv2_block3_0_bn[0][0] __________________________________________________________________________________________________ conv2_block3_1_conv (Conv2D) (None, None, None, 1 16384 conv2_block3_0_relu[0][0] __________________________________________________________________________________________________ conv2_block3_1_bn (BatchNormali (None, None, None, 1 512 conv2_block3_1_conv[0][0] __________________________________________________________________________________________________ conv2_block3_1_relu (Activation (None, None, None, 1 0 conv2_block3_1_bn[0][0] __________________________________________________________________________________________________ conv2_block3_2_conv (Conv2D) (None, None, None, 3 36864 conv2_block3_1_relu[0][0] __________________________________________________________________________________________________ conv2_block3_concat (Concatenat (None, None, None, 1 0 conv2_block2_concat[0][0] conv2_block3_2_conv[0][0] __________________________________________________________________________________________________ conv2_block4_0_bn (BatchNormali (None, None, None, 1 640 conv2_block3_concat[0][0] __________________________________________________________________________________________________ conv2_block4_0_relu (Activation (None, None, None, 1 0 conv2_block4_0_bn[0][0] __________________________________________________________________________________________________ conv2_block4_1_conv (Conv2D) (None, None, None, 1 20480 conv2_block4_0_relu[0][0] __________________________________________________________________________________________________ conv2_block4_1_bn (BatchNormali (None, None, None, 1 512 conv2_block4_1_conv[0][0] __________________________________________________________________________________________________ conv2_block4_1_relu (Activation (None, None, None, 1 0 conv2_block4_1_bn[0][0] __________________________________________________________________________________________________ conv2_block4_2_conv (Conv2D) (None, None, None, 3 36864 conv2_block4_1_relu[0][0] __________________________________________________________________________________________________ conv2_block4_concat (Concatenat (None, None, None, 1 0 conv2_block3_concat[0][0] conv2_block4_2_conv[0][0] __________________________________________________________________________________________________ conv2_block5_0_bn (BatchNormali (None, None, None, 1 768 conv2_block4_concat[0][0] __________________________________________________________________________________________________ conv2_block5_0_relu (Activation (None, None, None, 1 0 conv2_block5_0_bn[0][0] __________________________________________________________________________________________________ conv2_block5_1_conv (Conv2D) (None, None, None, 1 24576 conv2_block5_0_relu[0][0] __________________________________________________________________________________________________ conv2_block5_1_bn (BatchNormali (None, None, None, 1 512 conv2_block5_1_conv[0][0] __________________________________________________________________________________________________ conv2_block5_1_relu (Activation (None, None, None, 1 0 conv2_block5_1_bn[0][0] __________________________________________________________________________________________________ conv2_block5_2_conv (Conv2D) (None, None, None, 3 36864 conv2_block5_1_relu[0][0] __________________________________________________________________________________________________ conv2_block5_concat (Concatenat (None, None, None, 2 0 conv2_block4_concat[0][0] conv2_block5_2_conv[0][0] __________________________________________________________________________________________________ conv2_block6_0_bn (BatchNormali (None, None, None, 2 896 conv2_block5_concat[0][0] __________________________________________________________________________________________________ conv2_block6_0_relu (Activation (None, None, None, 2 0 conv2_block6_0_bn[0][0] __________________________________________________________________________________________________ conv2_block6_1_conv (Conv2D) (None, None, None, 1 28672 conv2_block6_0_relu[0][0] __________________________________________________________________________________________________ conv2_block6_1_bn (BatchNormali (None, None, None, 1 512 conv2_block6_1_conv[0][0] __________________________________________________________________________________________________ conv2_block6_1_relu (Activation (None, None, None, 1 0 conv2_block6_1_bn[0][0] __________________________________________________________________________________________________ conv2_block6_2_conv (Conv2D) (None, None, None, 3 36864 conv2_block6_1_relu[0][0] __________________________________________________________________________________________________ conv2_block6_concat (Concatenat (None, None, None, 2 0 conv2_block5_concat[0][0] conv2_block6_2_conv[0][0] __________________________________________________________________________________________________ pool2_bn (BatchNormalization) (None, None, None, 2 1024 conv2_block6_concat[0][0] __________________________________________________________________________________________________ pool2_relu (Activation) (None, None, None, 2 0 pool2_bn[0][0] __________________________________________________________________________________________________ pool2_conv (Conv2D) (None, None, None, 1 32768 pool2_relu[0][0] __________________________________________________________________________________________________ pool2_pool (AveragePooling2D) (None, None, None, 1 0 pool2_conv[0][0] __________________________________________________________________________________________________ conv3_block1_0_bn (BatchNormali (None, None, None, 1 512 pool2_pool[0][0] __________________________________________________________________________________________________ conv3_block1_0_relu (Activation (None, None, None, 1 0 conv3_block1_0_bn[0][0] __________________________________________________________________________________________________ conv3_block1_1_conv (Conv2D) (None, None, None, 1 16384 conv3_block1_0_relu[0][0] __________________________________________________________________________________________________ conv3_block1_1_bn (BatchNormali (None, None, None, 1 512 conv3_block1_1_conv[0][0] __________________________________________________________________________________________________ conv3_block1_1_relu (Activation (None, None, None, 1 0 conv3_block1_1_bn[0][0] __________________________________________________________________________________________________ conv3_block1_2_conv (Conv2D) (None, None, None, 3 36864 conv3_block1_1_relu[0][0] __________________________________________________________________________________________________ conv3_block1_concat (Concatenat (None, None, None, 1 0 pool2_pool[0][0] conv3_block1_2_conv[0][0] __________________________________________________________________________________________________ conv3_block2_0_bn (BatchNormali (None, None, None, 1 640 conv3_block1_concat[0][0] __________________________________________________________________________________________________ conv3_block2_0_relu (Activation (None, None, None, 1 0 conv3_block2_0_bn[0][0] __________________________________________________________________________________________________ conv3_block2_1_conv (Conv2D) (None, None, None, 1 20480 conv3_block2_0_relu[0][0] __________________________________________________________________________________________________ conv3_block2_1_bn (BatchNormali (None, None, None, 1 512 conv3_block2_1_conv[0][0] __________________________________________________________________________________________________ conv3_block2_1_relu (Activation (None, None, None, 1 0 conv3_block2_1_bn[0][0] __________________________________________________________________________________________________ conv3_block2_2_conv (Conv2D) (None, None, None, 3 36864 conv3_block2_1_relu[0][0] __________________________________________________________________________________________________ conv3_block2_concat (Concatenat (None, None, None, 1 0 conv3_block1_concat[0][0] conv3_block2_2_conv[0][0] __________________________________________________________________________________________________ conv3_block3_0_bn (BatchNormali (None, None, None, 1 768 conv3_block2_concat[0][0] __________________________________________________________________________________________________ conv3_block3_0_relu (Activation (None, None, None, 1 0 conv3_block3_0_bn[0][0] __________________________________________________________________________________________________ conv3_block3_1_conv (Conv2D) (None, None, None, 1 24576 conv3_block3_0_relu[0][0] __________________________________________________________________________________________________ conv3_block3_1_bn (BatchNormali (None, None, None, 1 512 conv3_block3_1_conv[0][0] __________________________________________________________________________________________________ conv3_block3_1_relu (Activation (None, None, None, 1 0 conv3_block3_1_bn[0][0] __________________________________________________________________________________________________ conv3_block3_2_conv (Conv2D) (None, None, None, 3 36864 conv3_block3_1_relu[0][0] __________________________________________________________________________________________________ conv3_block3_concat (Concatenat (None, None, None, 2 0 conv3_block2_concat[0][0] conv3_block3_2_conv[0][0] __________________________________________________________________________________________________ conv3_block4_0_bn (BatchNormali (None, None, None, 2 896 conv3_block3_concat[0][0] __________________________________________________________________________________________________ conv3_block4_0_relu (Activation (None, None, None, 2 0 conv3_block4_0_bn[0][0] __________________________________________________________________________________________________ conv3_block4_1_conv (Conv2D) (None, None, None, 1 28672 conv3_block4_0_relu[0][0] __________________________________________________________________________________________________ conv3_block4_1_bn (BatchNormali (None, None, None, 1 512 conv3_block4_1_conv[0][0] __________________________________________________________________________________________________ conv3_block4_1_relu (Activation (None, None, None, 1 0 conv3_block4_1_bn[0][0] __________________________________________________________________________________________________ conv3_block4_2_conv (Conv2D) (None, None, None, 3 36864 conv3_block4_1_relu[0][0] __________________________________________________________________________________________________ conv3_block4_concat (Concatenat (None, None, None, 2 0 conv3_block3_concat[0][0] conv3_block4_2_conv[0][0] __________________________________________________________________________________________________ conv3_block5_0_bn (BatchNormali (None, None, None, 2 1024 conv3_block4_concat[0][0] __________________________________________________________________________________________________ conv3_block5_0_relu (Activation (None, None, None, 2 0 conv3_block5_0_bn[0][0] __________________________________________________________________________________________________ conv3_block5_1_conv (Conv2D) (None, None, None, 1 32768 conv3_block5_0_relu[0][0] __________________________________________________________________________________________________ conv3_block5_1_bn (BatchNormali (None, None, None, 1 512 conv3_block5_1_conv[0][0] __________________________________________________________________________________________________ conv3_block5_1_relu (Activation (None, None, None, 1 0 conv3_block5_1_bn[0][0] __________________________________________________________________________________________________ conv3_block5_2_conv (Conv2D) (None, None, None, 3 36864 conv3_block5_1_relu[0][0] __________________________________________________________________________________________________ conv3_block5_concat (Concatenat (None, None, None, 2 0 conv3_block4_concat[0][0] conv3_block5_2_conv[0][0] __________________________________________________________________________________________________ conv3_block6_0_bn (BatchNormali (None, None, None, 2 1152 conv3_block5_concat[0][0] __________________________________________________________________________________________________ conv3_block6_0_relu (Activation (None, None, None, 2 0 conv3_block6_0_bn[0][0] __________________________________________________________________________________________________ conv3_block6_1_conv (Conv2D) (None, None, None, 1 36864 conv3_block6_0_relu[0][0] __________________________________________________________________________________________________ conv3_block6_1_bn (BatchNormali (None, None, None, 1 512 conv3_block6_1_conv[0][0] __________________________________________________________________________________________________ conv3_block6_1_relu (Activation (None, None, None, 1 0 conv3_block6_1_bn[0][0] __________________________________________________________________________________________________ conv3_block6_2_conv (Conv2D) (None, None, None, 3 36864 conv3_block6_1_relu[0][0] __________________________________________________________________________________________________ conv3_block6_concat (Concatenat (None, None, None, 3 0 conv3_block5_concat[0][0] conv3_block6_2_conv[0][0] __________________________________________________________________________________________________ conv3_block7_0_bn (BatchNormali (None, None, None, 3 1280 conv3_block6_concat[0][0] __________________________________________________________________________________________________ conv3_block7_0_relu (Activation (None, None, None, 3 0 conv3_block7_0_bn[0][0] __________________________________________________________________________________________________ conv3_block7_1_conv (Conv2D) (None, None, None, 1 40960 conv3_block7_0_relu[0][0] __________________________________________________________________________________________________ conv3_block7_1_bn (BatchNormali (None, None, None, 1 512 conv3_block7_1_conv[0][0] __________________________________________________________________________________________________ conv3_block7_1_relu (Activation (None, None, None, 1 0 conv3_block7_1_bn[0][0] __________________________________________________________________________________________________ conv3_block7_2_conv (Conv2D) (None, None, None, 3 36864 conv3_block7_1_relu[0][0] __________________________________________________________________________________________________ conv3_block7_concat (Concatenat (None, None, None, 3 0 conv3_block6_concat[0][0] conv3_block7_2_conv[0][0] __________________________________________________________________________________________________ conv3_block8_0_bn (BatchNormali (None, None, None, 3 1408 conv3_block7_concat[0][0] __________________________________________________________________________________________________ conv3_block8_0_relu (Activation (None, None, None, 3 0 conv3_block8_0_bn[0][0] __________________________________________________________________________________________________ conv3_block8_1_conv (Conv2D) (None, None, None, 1 45056 conv3_block8_0_relu[0][0] __________________________________________________________________________________________________ conv3_block8_1_bn (BatchNormali (None, None, None, 1 512 conv3_block8_1_conv[0][0] __________________________________________________________________________________________________ conv3_block8_1_relu (Activation (None, None, None, 1 0 conv3_block8_1_bn[0][0] __________________________________________________________________________________________________ conv3_block8_2_conv (Conv2D) (None, None, None, 3 36864 conv3_block8_1_relu[0][0] __________________________________________________________________________________________________ conv3_block8_concat (Concatenat (None, None, None, 3 0 conv3_block7_concat[0][0] conv3_block8_2_conv[0][0] __________________________________________________________________________________________________ conv3_block9_0_bn (BatchNormali (None, None, None, 3 1536 conv3_block8_concat[0][0] __________________________________________________________________________________________________ conv3_block9_0_relu (Activation (None, None, None, 3 0 conv3_block9_0_bn[0][0] __________________________________________________________________________________________________ conv3_block9_1_conv (Conv2D) (None, None, None, 1 49152 conv3_block9_0_relu[0][0] __________________________________________________________________________________________________ conv3_block9_1_bn (BatchNormali (None, None, None, 1 512 conv3_block9_1_conv[0][0] __________________________________________________________________________________________________ conv3_block9_1_relu (Activation (None, None, None, 1 0 conv3_block9_1_bn[0][0] __________________________________________________________________________________________________ conv3_block9_2_conv (Conv2D) (None, None, None, 3 36864 conv3_block9_1_relu[0][0] __________________________________________________________________________________________________ conv3_block9_concat (Concatenat (None, None, None, 4 0 conv3_block8_concat[0][0] conv3_block9_2_conv[0][0] __________________________________________________________________________________________________ conv3_block10_0_bn (BatchNormal (None, None, None, 4 1664 conv3_block9_concat[0][0] __________________________________________________________________________________________________ conv3_block10_0_relu (Activatio (None, None, None, 4 0 conv3_block10_0_bn[0][0] __________________________________________________________________________________________________ conv3_block10_1_conv (Conv2D) (None, None, None, 1 53248 conv3_block10_0_relu[0][0] __________________________________________________________________________________________________ conv3_block10_1_bn (BatchNormal (None, None, None, 1 512 conv3_block10_1_conv[0][0] __________________________________________________________________________________________________ conv3_block10_1_relu (Activatio (None, None, None, 1 0 conv3_block10_1_bn[0][0] __________________________________________________________________________________________________ conv3_block10_2_conv (Conv2D) (None, None, None, 3 36864 conv3_block10_1_relu[0][0] __________________________________________________________________________________________________ conv3_block10_concat (Concatena (None, None, None, 4 0 conv3_block9_concat[0][0] conv3_block10_2_conv[0][0] __________________________________________________________________________________________________ conv3_block11_0_bn (BatchNormal (None, None, None, 4 1792 conv3_block10_concat[0][0] __________________________________________________________________________________________________ conv3_block11_0_relu (Activatio (None, None, None, 4 0 conv3_block11_0_bn[0][0] __________________________________________________________________________________________________ conv3_block11_1_conv (Conv2D) (None, None, None, 1 57344 conv3_block11_0_relu[0][0] __________________________________________________________________________________________________ conv3_block11_1_bn (BatchNormal (None, None, None, 1 512 conv3_block11_1_conv[0][0] __________________________________________________________________________________________________ conv3_block11_1_relu (Activatio (None, None, None, 1 0 conv3_block11_1_bn[0][0] __________________________________________________________________________________________________ conv3_block11_2_conv (Conv2D) (None, None, None, 3 36864 conv3_block11_1_relu[0][0] __________________________________________________________________________________________________ conv3_block11_concat (Concatena (None, None, None, 4 0 conv3_block10_concat[0][0] conv3_block11_2_conv[0][0] __________________________________________________________________________________________________ conv3_block12_0_bn (BatchNormal (None, None, None, 4 1920 conv3_block11_concat[0][0] __________________________________________________________________________________________________ conv3_block12_0_relu (Activatio (None, None, None, 4 0 conv3_block12_0_bn[0][0] __________________________________________________________________________________________________ conv3_block12_1_conv (Conv2D) (None, None, None, 1 61440 conv3_block12_0_relu[0][0] __________________________________________________________________________________________________ conv3_block12_1_bn (BatchNormal (None, None, None, 1 512 conv3_block12_1_conv[0][0] __________________________________________________________________________________________________ conv3_block12_1_relu (Activatio (None, None, None, 1 0 conv3_block12_1_bn[0][0] __________________________________________________________________________________________________ conv3_block12_2_conv (Conv2D) (None, None, None, 3 36864 conv3_block12_1_relu[0][0] __________________________________________________________________________________________________ conv3_block12_concat (Concatena (None, None, None, 5 0 conv3_block11_concat[0][0] conv3_block12_2_conv[0][0] __________________________________________________________________________________________________ pool3_bn (BatchNormalization) (None, None, None, 5 2048 conv3_block12_concat[0][0] __________________________________________________________________________________________________ pool3_relu (Activation) (None, None, None, 5 0 pool3_bn[0][0] __________________________________________________________________________________________________ pool3_conv (Conv2D) (None, None, None, 2 131072 pool3_relu[0][0] __________________________________________________________________________________________________ pool3_pool (AveragePooling2D) (None, None, None, 2 0 pool3_conv[0][0] __________________________________________________________________________________________________ conv4_block1_0_bn (BatchNormali (None, None, None, 2 1024 pool3_pool[0][0] __________________________________________________________________________________________________ conv4_block1_0_relu (Activation (None, None, None, 2 0 conv4_block1_0_bn[0][0] __________________________________________________________________________________________________ conv4_block1_1_conv (Conv2D) (None, None, None, 1 32768 conv4_block1_0_relu[0][0] __________________________________________________________________________________________________ conv4_block1_1_bn (BatchNormali (None, None, None, 1 512 conv4_block1_1_conv[0][0] __________________________________________________________________________________________________ conv4_block1_1_relu (Activation (None, None, None, 1 0 conv4_block1_1_bn[0][0] __________________________________________________________________________________________________ conv4_block1_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block1_1_relu[0][0] __________________________________________________________________________________________________ conv4_block1_concat (Concatenat (None, None, None, 2 0 pool3_pool[0][0] conv4_block1_2_conv[0][0] __________________________________________________________________________________________________ conv4_block2_0_bn (BatchNormali (None, None, None, 2 1152 conv4_block1_concat[0][0] __________________________________________________________________________________________________ conv4_block2_0_relu (Activation (None, None, None, 2 0 conv4_block2_0_bn[0][0] __________________________________________________________________________________________________ conv4_block2_1_conv (Conv2D) (None, None, None, 1 36864 conv4_block2_0_relu[0][0] __________________________________________________________________________________________________ conv4_block2_1_bn (BatchNormali (None, None, None, 1 512 conv4_block2_1_conv[0][0] __________________________________________________________________________________________________ conv4_block2_1_relu (Activation (None, None, None, 1 0 conv4_block2_1_bn[0][0] __________________________________________________________________________________________________ conv4_block2_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block2_1_relu[0][0] __________________________________________________________________________________________________ conv4_block2_concat (Concatenat (None, None, None, 3 0 conv4_block1_concat[0][0] conv4_block2_2_conv[0][0] __________________________________________________________________________________________________ conv4_block3_0_bn (BatchNormali (None, None, None, 3 1280 conv4_block2_concat[0][0] __________________________________________________________________________________________________ conv4_block3_0_relu (Activation (None, None, None, 3 0 conv4_block3_0_bn[0][0] __________________________________________________________________________________________________ conv4_block3_1_conv (Conv2D) (None, None, None, 1 40960 conv4_block3_0_relu[0][0] __________________________________________________________________________________________________ conv4_block3_1_bn (BatchNormali (None, None, None, 1 512 conv4_block3_1_conv[0][0] __________________________________________________________________________________________________ conv4_block3_1_relu (Activation (None, None, None, 1 0 conv4_block3_1_bn[0][0] __________________________________________________________________________________________________ conv4_block3_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block3_1_relu[0][0] __________________________________________________________________________________________________ conv4_block3_concat (Concatenat (None, None, None, 3 0 conv4_block2_concat[0][0] conv4_block3_2_conv[0][0] __________________________________________________________________________________________________ conv4_block4_0_bn (BatchNormali (None, None, None, 3 1408 conv4_block3_concat[0][0] __________________________________________________________________________________________________ conv4_block4_0_relu (Activation (None, None, None, 3 0 conv4_block4_0_bn[0][0] __________________________________________________________________________________________________ conv4_block4_1_conv (Conv2D) (None, None, None, 1 45056 conv4_block4_0_relu[0][0] __________________________________________________________________________________________________ conv4_block4_1_bn (BatchNormali (None, None, None, 1 512 conv4_block4_1_conv[0][0] __________________________________________________________________________________________________ conv4_block4_1_relu (Activation (None, None, None, 1 0 conv4_block4_1_bn[0][0] __________________________________________________________________________________________________ conv4_block4_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block4_1_relu[0][0] __________________________________________________________________________________________________ conv4_block4_concat (Concatenat (None, None, None, 3 0 conv4_block3_concat[0][0] conv4_block4_2_conv[0][0] __________________________________________________________________________________________________ conv4_block5_0_bn (BatchNormali (None, None, None, 3 1536 conv4_block4_concat[0][0] __________________________________________________________________________________________________ conv4_block5_0_relu (Activation (None, None, None, 3 0 conv4_block5_0_bn[0][0] __________________________________________________________________________________________________ conv4_block5_1_conv (Conv2D) (None, None, None, 1 49152 conv4_block5_0_relu[0][0] __________________________________________________________________________________________________ conv4_block5_1_bn (BatchNormali (None, None, None, 1 512 conv4_block5_1_conv[0][0] __________________________________________________________________________________________________ conv4_block5_1_relu (Activation (None, None, None, 1 0 conv4_block5_1_bn[0][0] __________________________________________________________________________________________________ conv4_block5_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block5_1_relu[0][0] __________________________________________________________________________________________________ conv4_block5_concat (Concatenat (None, None, None, 4 0 conv4_block4_concat[0][0] conv4_block5_2_conv[0][0] __________________________________________________________________________________________________ conv4_block6_0_bn (BatchNormali (None, None, None, 4 1664 conv4_block5_concat[0][0] __________________________________________________________________________________________________ conv4_block6_0_relu (Activation (None, None, None, 4 0 conv4_block6_0_bn[0][0] __________________________________________________________________________________________________ conv4_block6_1_conv (Conv2D) (None, None, None, 1 53248 conv4_block6_0_relu[0][0] __________________________________________________________________________________________________ conv4_block6_1_bn (BatchNormali (None, None, None, 1 512 conv4_block6_1_conv[0][0] __________________________________________________________________________________________________ conv4_block6_1_relu (Activation (None, None, None, 1 0 conv4_block6_1_bn[0][0] __________________________________________________________________________________________________ conv4_block6_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block6_1_relu[0][0] __________________________________________________________________________________________________ conv4_block6_concat (Concatenat (None, None, None, 4 0 conv4_block5_concat[0][0] conv4_block6_2_conv[0][0] __________________________________________________________________________________________________ conv4_block7_0_bn (BatchNormali (None, None, None, 4 1792 conv4_block6_concat[0][0] __________________________________________________________________________________________________ conv4_block7_0_relu (Activation (None, None, None, 4 0 conv4_block7_0_bn[0][0] __________________________________________________________________________________________________ conv4_block7_1_conv (Conv2D) (None, None, None, 1 57344 conv4_block7_0_relu[0][0] __________________________________________________________________________________________________ conv4_block7_1_bn (BatchNormali (None, None, None, 1 512 conv4_block7_1_conv[0][0] __________________________________________________________________________________________________ conv4_block7_1_relu (Activation (None, None, None, 1 0 conv4_block7_1_bn[0][0] __________________________________________________________________________________________________ conv4_block7_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block7_1_relu[0][0] __________________________________________________________________________________________________ conv4_block7_concat (Concatenat (None, None, None, 4 0 conv4_block6_concat[0][0] conv4_block7_2_conv[0][0] __________________________________________________________________________________________________ conv4_block8_0_bn (BatchNormali (None, None, None, 4 1920 conv4_block7_concat[0][0] __________________________________________________________________________________________________ conv4_block8_0_relu (Activation (None, None, None, 4 0 conv4_block8_0_bn[0][0] __________________________________________________________________________________________________ conv4_block8_1_conv (Conv2D) (None, None, None, 1 61440 conv4_block8_0_relu[0][0] __________________________________________________________________________________________________ conv4_block8_1_bn (BatchNormali (None, None, None, 1 512 conv4_block8_1_conv[0][0] __________________________________________________________________________________________________ conv4_block8_1_relu (Activation (None, None, None, 1 0 conv4_block8_1_bn[0][0] __________________________________________________________________________________________________ conv4_block8_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block8_1_relu[0][0] __________________________________________________________________________________________________ conv4_block8_concat (Concatenat (None, None, None, 5 0 conv4_block7_concat[0][0] conv4_block8_2_conv[0][0] __________________________________________________________________________________________________ conv4_block9_0_bn (BatchNormali (None, None, None, 5 2048 conv4_block8_concat[0][0] __________________________________________________________________________________________________ conv4_block9_0_relu (Activation (None, None, None, 5 0 conv4_block9_0_bn[0][0] __________________________________________________________________________________________________ conv4_block9_1_conv (Conv2D) (None, None, None, 1 65536 conv4_block9_0_relu[0][0] __________________________________________________________________________________________________ conv4_block9_1_bn (BatchNormali (None, None, None, 1 512 conv4_block9_1_conv[0][0] __________________________________________________________________________________________________ conv4_block9_1_relu (Activation (None, None, None, 1 0 conv4_block9_1_bn[0][0] __________________________________________________________________________________________________ conv4_block9_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block9_1_relu[0][0] __________________________________________________________________________________________________ conv4_block9_concat (Concatenat (None, None, None, 5 0 conv4_block8_concat[0][0] conv4_block9_2_conv[0][0] __________________________________________________________________________________________________ conv4_block10_0_bn (BatchNormal (None, None, None, 5 2176 conv4_block9_concat[0][0] __________________________________________________________________________________________________ conv4_block10_0_relu (Activatio (None, None, None, 5 0 conv4_block10_0_bn[0][0] __________________________________________________________________________________________________ conv4_block10_1_conv (Conv2D) (None, None, None, 1 69632 conv4_block10_0_relu[0][0] __________________________________________________________________________________________________ conv4_block10_1_bn (BatchNormal (None, None, None, 1 512 conv4_block10_1_conv[0][0] __________________________________________________________________________________________________ conv4_block10_1_relu (Activatio (None, None, None, 1 0 conv4_block10_1_bn[0][0] __________________________________________________________________________________________________ conv4_block10_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block10_1_relu[0][0] __________________________________________________________________________________________________ conv4_block10_concat (Concatena (None, None, None, 5 0 conv4_block9_concat[0][0] conv4_block10_2_conv[0][0] __________________________________________________________________________________________________ conv4_block11_0_bn (BatchNormal (None, None, None, 5 2304 conv4_block10_concat[0][0] __________________________________________________________________________________________________ conv4_block11_0_relu (Activatio (None, None, None, 5 0 conv4_block11_0_bn[0][0] __________________________________________________________________________________________________ conv4_block11_1_conv (Conv2D) (None, None, None, 1 73728 conv4_block11_0_relu[0][0] __________________________________________________________________________________________________ conv4_block11_1_bn (BatchNormal (None, None, None, 1 512 conv4_block11_1_conv[0][0] __________________________________________________________________________________________________ conv4_block11_1_relu (Activatio (None, None, None, 1 0 conv4_block11_1_bn[0][0] __________________________________________________________________________________________________ conv4_block11_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block11_1_relu[0][0] __________________________________________________________________________________________________ conv4_block11_concat (Concatena (None, None, None, 6 0 conv4_block10_concat[0][0] conv4_block11_2_conv[0][0] __________________________________________________________________________________________________ conv4_block12_0_bn (BatchNormal (None, None, None, 6 2432 conv4_block11_concat[0][0] __________________________________________________________________________________________________ conv4_block12_0_relu (Activatio (None, None, None, 6 0 conv4_block12_0_bn[0][0] __________________________________________________________________________________________________ conv4_block12_1_conv (Conv2D) (None, None, None, 1 77824 conv4_block12_0_relu[0][0] __________________________________________________________________________________________________ conv4_block12_1_bn (BatchNormal (None, None, None, 1 512 conv4_block12_1_conv[0][0] __________________________________________________________________________________________________ conv4_block12_1_relu (Activatio (None, None, None, 1 0 conv4_block12_1_bn[0][0] __________________________________________________________________________________________________ conv4_block12_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block12_1_relu[0][0] __________________________________________________________________________________________________ conv4_block12_concat (Concatena (None, None, None, 6 0 conv4_block11_concat[0][0] conv4_block12_2_conv[0][0] __________________________________________________________________________________________________ conv4_block13_0_bn (BatchNormal (None, None, None, 6 2560 conv4_block12_concat[0][0] __________________________________________________________________________________________________ conv4_block13_0_relu (Activatio (None, None, None, 6 0 conv4_block13_0_bn[0][0] __________________________________________________________________________________________________ conv4_block13_1_conv (Conv2D) (None, None, None, 1 81920 conv4_block13_0_relu[0][0] __________________________________________________________________________________________________ conv4_block13_1_bn (BatchNormal (None, None, None, 1 512 conv4_block13_1_conv[0][0] __________________________________________________________________________________________________ conv4_block13_1_relu (Activatio (None, None, None, 1 0 conv4_block13_1_bn[0][0] __________________________________________________________________________________________________ conv4_block13_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block13_1_relu[0][0] __________________________________________________________________________________________________ conv4_block13_concat (Concatena (None, None, None, 6 0 conv4_block12_concat[0][0] conv4_block13_2_conv[0][0] __________________________________________________________________________________________________ conv4_block14_0_bn (BatchNormal (None, None, None, 6 2688 conv4_block13_concat[0][0] __________________________________________________________________________________________________ conv4_block14_0_relu (Activatio (None, None, None, 6 0 conv4_block14_0_bn[0][0] __________________________________________________________________________________________________ conv4_block14_1_conv (Conv2D) (None, None, None, 1 86016 conv4_block14_0_relu[0][0] __________________________________________________________________________________________________ conv4_block14_1_bn (BatchNormal (None, None, None, 1 512 conv4_block14_1_conv[0][0] __________________________________________________________________________________________________ conv4_block14_1_relu (Activatio (None, None, None, 1 0 conv4_block14_1_bn[0][0] __________________________________________________________________________________________________ conv4_block14_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block14_1_relu[0][0] __________________________________________________________________________________________________ conv4_block14_concat (Concatena (None, None, None, 7 0 conv4_block13_concat[0][0] conv4_block14_2_conv[0][0] __________________________________________________________________________________________________ conv4_block15_0_bn (BatchNormal (None, None, None, 7 2816 conv4_block14_concat[0][0] __________________________________________________________________________________________________ conv4_block15_0_relu (Activatio (None, None, None, 7 0 conv4_block15_0_bn[0][0] __________________________________________________________________________________________________ conv4_block15_1_conv (Conv2D) (None, None, None, 1 90112 conv4_block15_0_relu[0][0] __________________________________________________________________________________________________ conv4_block15_1_bn (BatchNormal (None, None, None, 1 512 conv4_block15_1_conv[0][0] __________________________________________________________________________________________________ conv4_block15_1_relu (Activatio (None, None, None, 1 0 conv4_block15_1_bn[0][0] __________________________________________________________________________________________________ conv4_block15_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block15_1_relu[0][0] __________________________________________________________________________________________________ conv4_block15_concat (Concatena (None, None, None, 7 0 conv4_block14_concat[0][0] conv4_block15_2_conv[0][0] __________________________________________________________________________________________________ conv4_block16_0_bn (BatchNormal (None, None, None, 7 2944 conv4_block15_concat[0][0] __________________________________________________________________________________________________ conv4_block16_0_relu (Activatio (None, None, None, 7 0 conv4_block16_0_bn[0][0] __________________________________________________________________________________________________ conv4_block16_1_conv (Conv2D) (None, None, None, 1 94208 conv4_block16_0_relu[0][0] __________________________________________________________________________________________________ conv4_block16_1_bn (BatchNormal (None, None, None, 1 512 conv4_block16_1_conv[0][0] __________________________________________________________________________________________________ conv4_block16_1_relu (Activatio (None, None, None, 1 0 conv4_block16_1_bn[0][0] __________________________________________________________________________________________________ conv4_block16_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block16_1_relu[0][0] __________________________________________________________________________________________________ conv4_block16_concat (Concatena (None, None, None, 7 0 conv4_block15_concat[0][0] conv4_block16_2_conv[0][0] __________________________________________________________________________________________________ conv4_block17_0_bn (BatchNormal (None, None, None, 7 3072 conv4_block16_concat[0][0] __________________________________________________________________________________________________ conv4_block17_0_relu (Activatio (None, None, None, 7 0 conv4_block17_0_bn[0][0] __________________________________________________________________________________________________ conv4_block17_1_conv (Conv2D) (None, None, None, 1 98304 conv4_block17_0_relu[0][0] __________________________________________________________________________________________________ conv4_block17_1_bn (BatchNormal (None, None, None, 1 512 conv4_block17_1_conv[0][0] __________________________________________________________________________________________________ conv4_block17_1_relu (Activatio (None, None, None, 1 0 conv4_block17_1_bn[0][0] __________________________________________________________________________________________________ conv4_block17_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block17_1_relu[0][0] __________________________________________________________________________________________________ conv4_block17_concat (Concatena (None, None, None, 8 0 conv4_block16_concat[0][0] conv4_block17_2_conv[0][0] __________________________________________________________________________________________________ conv4_block18_0_bn (BatchNormal (None, None, None, 8 3200 conv4_block17_concat[0][0] __________________________________________________________________________________________________ conv4_block18_0_relu (Activatio (None, None, None, 8 0 conv4_block18_0_bn[0][0] __________________________________________________________________________________________________ conv4_block18_1_conv (Conv2D) (None, None, None, 1 102400 conv4_block18_0_relu[0][0] __________________________________________________________________________________________________ conv4_block18_1_bn (BatchNormal (None, None, None, 1 512 conv4_block18_1_conv[0][0] __________________________________________________________________________________________________ conv4_block18_1_relu (Activatio (None, None, None, 1 0 conv4_block18_1_bn[0][0] __________________________________________________________________________________________________ conv4_block18_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block18_1_relu[0][0] __________________________________________________________________________________________________ conv4_block18_concat (Concatena (None, None, None, 8 0 conv4_block17_concat[0][0] conv4_block18_2_conv[0][0] __________________________________________________________________________________________________ conv4_block19_0_bn (BatchNormal (None, None, None, 8 3328 conv4_block18_concat[0][0] __________________________________________________________________________________________________ conv4_block19_0_relu (Activatio (None, None, None, 8 0 conv4_block19_0_bn[0][0] __________________________________________________________________________________________________ conv4_block19_1_conv (Conv2D) (None, None, None, 1 106496 conv4_block19_0_relu[0][0] __________________________________________________________________________________________________ conv4_block19_1_bn (BatchNormal (None, None, None, 1 512 conv4_block19_1_conv[0][0] __________________________________________________________________________________________________ conv4_block19_1_relu (Activatio (None, None, None, 1 0 conv4_block19_1_bn[0][0] __________________________________________________________________________________________________ conv4_block19_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block19_1_relu[0][0] __________________________________________________________________________________________________ conv4_block19_concat (Concatena (None, None, None, 8 0 conv4_block18_concat[0][0] conv4_block19_2_conv[0][0] __________________________________________________________________________________________________ conv4_block20_0_bn (BatchNormal (None, None, None, 8 3456 conv4_block19_concat[0][0] __________________________________________________________________________________________________ conv4_block20_0_relu (Activatio (None, None, None, 8 0 conv4_block20_0_bn[0][0] __________________________________________________________________________________________________ conv4_block20_1_conv (Conv2D) (None, None, None, 1 110592 conv4_block20_0_relu[0][0] __________________________________________________________________________________________________ conv4_block20_1_bn (BatchNormal (None, None, None, 1 512 conv4_block20_1_conv[0][0] __________________________________________________________________________________________________ conv4_block20_1_relu (Activatio (None, None, None, 1 0 conv4_block20_1_bn[0][0] __________________________________________________________________________________________________ conv4_block20_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block20_1_relu[0][0] __________________________________________________________________________________________________ conv4_block20_concat (Concatena (None, None, None, 8 0 conv4_block19_concat[0][0] conv4_block20_2_conv[0][0] __________________________________________________________________________________________________ conv4_block21_0_bn (BatchNormal (None, None, None, 8 3584 conv4_block20_concat[0][0] __________________________________________________________________________________________________ conv4_block21_0_relu (Activatio (None, None, None, 8 0 conv4_block21_0_bn[0][0] __________________________________________________________________________________________________ conv4_block21_1_conv (Conv2D) (None, None, None, 1 114688 conv4_block21_0_relu[0][0] __________________________________________________________________________________________________ conv4_block21_1_bn (BatchNormal (None, None, None, 1 512 conv4_block21_1_conv[0][0] __________________________________________________________________________________________________ conv4_block21_1_relu (Activatio (None, None, None, 1 0 conv4_block21_1_bn[0][0] __________________________________________________________________________________________________ conv4_block21_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block21_1_relu[0][0] __________________________________________________________________________________________________ conv4_block21_concat (Concatena (None, None, None, 9 0 conv4_block20_concat[0][0] conv4_block21_2_conv[0][0] __________________________________________________________________________________________________ conv4_block22_0_bn (BatchNormal (None, None, None, 9 3712 conv4_block21_concat[0][0] __________________________________________________________________________________________________ conv4_block22_0_relu (Activatio (None, None, None, 9 0 conv4_block22_0_bn[0][0] __________________________________________________________________________________________________ conv4_block22_1_conv (Conv2D) (None, None, None, 1 118784 conv4_block22_0_relu[0][0] __________________________________________________________________________________________________ conv4_block22_1_bn (BatchNormal (None, None, None, 1 512 conv4_block22_1_conv[0][0] __________________________________________________________________________________________________ conv4_block22_1_relu (Activatio (None, None, None, 1 0 conv4_block22_1_bn[0][0] __________________________________________________________________________________________________ conv4_block22_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block22_1_relu[0][0] __________________________________________________________________________________________________ conv4_block22_concat (Concatena (None, None, None, 9 0 conv4_block21_concat[0][0] conv4_block22_2_conv[0][0] __________________________________________________________________________________________________ conv4_block23_0_bn (BatchNormal (None, None, None, 9 3840 conv4_block22_concat[0][0] __________________________________________________________________________________________________ conv4_block23_0_relu (Activatio (None, None, None, 9 0 conv4_block23_0_bn[0][0] __________________________________________________________________________________________________ conv4_block23_1_conv (Conv2D) (None, None, None, 1 122880 conv4_block23_0_relu[0][0] __________________________________________________________________________________________________ conv4_block23_1_bn (BatchNormal (None, None, None, 1 512 conv4_block23_1_conv[0][0] __________________________________________________________________________________________________ conv4_block23_1_relu (Activatio (None, None, None, 1 0 conv4_block23_1_bn[0][0] __________________________________________________________________________________________________ conv4_block23_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block23_1_relu[0][0] __________________________________________________________________________________________________ conv4_block23_concat (Concatena (None, None, None, 9 0 conv4_block22_concat[0][0] conv4_block23_2_conv[0][0] __________________________________________________________________________________________________ conv4_block24_0_bn (BatchNormal (None, None, None, 9 3968 conv4_block23_concat[0][0] __________________________________________________________________________________________________ conv4_block24_0_relu (Activatio (None, None, None, 9 0 conv4_block24_0_bn[0][0] __________________________________________________________________________________________________ conv4_block24_1_conv (Conv2D) (None, None, None, 1 126976 conv4_block24_0_relu[0][0] __________________________________________________________________________________________________ conv4_block24_1_bn (BatchNormal (None, None, None, 1 512 conv4_block24_1_conv[0][0] __________________________________________________________________________________________________ conv4_block24_1_relu (Activatio (None, None, None, 1 0 conv4_block24_1_bn[0][0] __________________________________________________________________________________________________ conv4_block24_2_conv (Conv2D) (None, None, None, 3 36864 conv4_block24_1_relu[0][0] __________________________________________________________________________________________________ conv4_block24_concat (Concatena (None, None, None, 1 0 conv4_block23_concat[0][0] conv4_block24_2_conv[0][0] __________________________________________________________________________________________________ pool4_bn (BatchNormalization) (None, None, None, 1 4096 conv4_block24_concat[0][0] __________________________________________________________________________________________________ pool4_relu (Activation) (None, None, None, 1 0 pool4_bn[0][0] __________________________________________________________________________________________________ pool4_conv (Conv2D) (None, None, None, 5 524288 pool4_relu[0][0] __________________________________________________________________________________________________ pool4_pool (AveragePooling2D) (None, None, None, 5 0 pool4_conv[0][0] __________________________________________________________________________________________________ conv5_block1_0_bn (BatchNormali (None, None, None, 5 2048 pool4_pool[0][0] __________________________________________________________________________________________________ conv5_block1_0_relu (Activation (None, None, None, 5 0 conv5_block1_0_bn[0][0] __________________________________________________________________________________________________ conv5_block1_1_conv (Conv2D) (None, None, None, 1 65536 conv5_block1_0_relu[0][0] __________________________________________________________________________________________________ conv5_block1_1_bn (BatchNormali (None, None, None, 1 512 conv5_block1_1_conv[0][0] __________________________________________________________________________________________________ conv5_block1_1_relu (Activation (None, None, None, 1 0 conv5_block1_1_bn[0][0] __________________________________________________________________________________________________ conv5_block1_2_conv (Conv2D) (None, None, None, 3 36864 conv5_block1_1_relu[0][0] __________________________________________________________________________________________________ conv5_block1_concat (Concatenat (None, None, None, 5 0 pool4_pool[0][0] conv5_block1_2_conv[0][0] __________________________________________________________________________________________________ conv5_block2_0_bn (BatchNormali (None, None, None, 5 2176 conv5_block1_concat[0][0] __________________________________________________________________________________________________ conv5_block2_0_relu (Activation (None, None, None, 5 0 conv5_block2_0_bn[0][0] __________________________________________________________________________________________________ conv5_block2_1_conv (Conv2D) (None, None, None, 1 69632 conv5_block2_0_relu[0][0] __________________________________________________________________________________________________ conv5_block2_1_bn (BatchNormali (None, None, None, 1 512 conv5_block2_1_conv[0][0] __________________________________________________________________________________________________ conv5_block2_1_relu (Activation (None, None, None, 1 0 conv5_block2_1_bn[0][0] __________________________________________________________________________________________________ conv5_block2_2_conv (Conv2D) (None, None, None, 3 36864 conv5_block2_1_relu[0][0] __________________________________________________________________________________________________ conv5_block2_concat (Concatenat (None, None, None, 5 0 conv5_block1_concat[0][0] conv5_block2_2_conv[0][0] __________________________________________________________________________________________________ conv5_block3_0_bn (BatchNormali (None, None, None, 5 2304 conv5_block2_concat[0][0] __________________________________________________________________________________________________ conv5_block3_0_relu (Activation (None, None, None, 5 0 conv5_block3_0_bn[0][0] __________________________________________________________________________________________________ conv5_block3_1_conv (Conv2D) (None, None, None, 1 73728 conv5_block3_0_relu[0][0] __________________________________________________________________________________________________ conv5_block3_1_bn (BatchNormali (None, None, None, 1 512 conv5_block3_1_conv[0][0] __________________________________________________________________________________________________ conv5_block3_1_relu (Activation (None, None, None, 1 0 conv5_block3_1_bn[0][0] __________________________________________________________________________________________________ conv5_block3_2_conv (Conv2D) (None, None, None, 3 36864 conv5_block3_1_relu[0][0] __________________________________________________________________________________________________ conv5_block3_concat (Concatenat (None, None, None, 6 0 conv5_block2_concat[0][0] conv5_block3_2_conv[0][0] __________________________________________________________________________________________________ conv5_block4_0_bn (BatchNormali (None, None, None, 6 2432 conv5_block3_concat[0][0] __________________________________________________________________________________________________ conv5_block4_0_relu (Activation (None, None, None, 6 0 conv5_block4_0_bn[0][0] __________________________________________________________________________________________________ conv5_block4_1_conv (Conv2D) (None, None, None, 1 77824 conv5_block4_0_relu[0][0] __________________________________________________________________________________________________ conv5_block4_1_bn (BatchNormali (None, None, None, 1 512 conv5_block4_1_conv[0][0] __________________________________________________________________________________________________ conv5_block4_1_relu (Activation (None, None, None, 1 0 conv5_block4_1_bn[0][0] __________________________________________________________________________________________________ conv5_block4_2_conv (Conv2D) (None, None, None, 3 36864 conv5_block4_1_relu[0][0] __________________________________________________________________________________________________ conv5_block4_concat (Concatenat (None, None, None, 6 0 conv5_block3_concat[0][0] conv5_block4_2_conv[0][0] __________________________________________________________________________________________________ conv5_block5_0_bn (BatchNormali (None, None, None, 6 2560 conv5_block4_concat[0][0] __________________________________________________________________________________________________ conv5_block5_0_relu (Activation (None, None, None, 6 0 conv5_block5_0_bn[0][0] __________________________________________________________________________________________________ conv5_block5_1_conv (Conv2D) (None, None, None, 1 81920 conv5_block5_0_relu[0][0] __________________________________________________________________________________________________ conv5_block5_1_bn (BatchNormali (None, None, None, 1 512 conv5_block5_1_conv[0][0] __________________________________________________________________________________________________ conv5_block5_1_relu (Activation (None, None, None, 1 0 conv5_block5_1_bn[0][0] __________________________________________________________________________________________________ conv5_block5_2_conv (Conv2D) (None, None, None, 3 36864 conv5_block5_1_relu[0][0] __________________________________________________________________________________________________ conv5_block5_concat (Concatenat (None, None, None, 6 0 conv5_block4_concat[0][0] conv5_block5_2_conv[0][0] __________________________________________________________________________________________________ conv5_block6_0_bn (BatchNormali (None, None, None, 6 2688 conv5_block5_concat[0][0] __________________________________________________________________________________________________ conv5_block6_0_relu (Activation (None, None, None, 6 0 conv5_block6_0_bn[0][0] __________________________________________________________________________________________________ conv5_block6_1_conv (Conv2D) (None, None, None, 1 86016 conv5_block6_0_relu[0][0] __________________________________________________________________________________________________ conv5_block6_1_bn (BatchNormali (None, None, None, 1 512 conv5_block6_1_conv[0][0] __________________________________________________________________________________________________ conv5_block6_1_relu (Activation (None, None, None, 1 0 conv5_block6_1_bn[0][0] __________________________________________________________________________________________________ conv5_block6_2_conv (Conv2D) (None, None, None, 3 36864 conv5_block6_1_relu[0][0] __________________________________________________________________________________________________ conv5_block6_concat (Concatenat (None, None, None, 7 0 conv5_block5_concat[0][0] conv5_block6_2_conv[0][0] __________________________________________________________________________________________________ conv5_block7_0_bn (BatchNormali (None, None, None, 7 2816 conv5_block6_concat[0][0] __________________________________________________________________________________________________ conv5_block7_0_relu (Activation (None, None, None, 7 0 conv5_block7_0_bn[0][0] __________________________________________________________________________________________________ conv5_block7_1_conv (Conv2D) (None, None, None, 1 90112 conv5_block7_0_relu[0][0] __________________________________________________________________________________________________ conv5_block7_1_bn (BatchNormali (None, None, None, 1 512 conv5_block7_1_conv[0][0] __________________________________________________________________________________________________ conv5_block7_1_relu (Activation (None, None, None, 1 0 conv5_block7_1_bn[0][0] __________________________________________________________________________________________________ conv5_block7_2_conv (Conv2D) (None, None, None, 3 36864 conv5_block7_1_relu[0][0] __________________________________________________________________________________________________ conv5_block7_concat (Concatenat (None, None, None, 7 0 conv5_block6_concat[0][0] conv5_block7_2_conv[0][0] __________________________________________________________________________________________________ conv5_block8_0_bn (BatchNormali (None, None, None, 7 2944 conv5_block7_concat[0][0] __________________________________________________________________________________________________ conv5_block8_0_relu (Activation (None, None, None, 7 0 conv5_block8_0_bn[0][0] __________________________________________________________________________________________________ conv5_block8_1_conv (Conv2D) (None, None, None, 1 94208 conv5_block8_0_relu[0][0] __________________________________________________________________________________________________ conv5_block8_1_bn (BatchNormali (None, None, None, 1 512 conv5_block8_1_conv[0][0] __________________________________________________________________________________________________ conv5_block8_1_relu (Activation (None, None, None, 1 0 conv5_block8_1_bn[0][0] __________________________________________________________________________________________________ conv5_block8_2_conv (Conv2D) (None, None, None, 3 36864 conv5_block8_1_relu[0][0] __________________________________________________________________________________________________ conv5_block8_concat (Concatenat (None, None, None, 7 0 conv5_block7_concat[0][0] conv5_block8_2_conv[0][0] __________________________________________________________________________________________________ conv5_block9_0_bn (BatchNormali (None, None, None, 7 3072 conv5_block8_concat[0][0] __________________________________________________________________________________________________ conv5_block9_0_relu (Activation (None, None, None, 7 0 conv5_block9_0_bn[0][0] __________________________________________________________________________________________________ conv5_block9_1_conv (Conv2D) (None, None, None, 1 98304 conv5_block9_0_relu[0][0] __________________________________________________________________________________________________ conv5_block9_1_bn (BatchNormali (None, None, None, 1 512 conv5_block9_1_conv[0][0] __________________________________________________________________________________________________ conv5_block9_1_relu (Activation (None, None, None, 1 0 conv5_block9_1_bn[0][0] __________________________________________________________________________________________________ conv5_block9_2_conv (Conv2D) (None, None, None, 3 36864 conv5_block9_1_relu[0][0] __________________________________________________________________________________________________ conv5_block9_concat (Concatenat (None, None, None, 8 0 conv5_block8_concat[0][0] conv5_block9_2_conv[0][0] __________________________________________________________________________________________________ conv5_block10_0_bn (BatchNormal (None, None, None, 8 3200 conv5_block9_concat[0][0] __________________________________________________________________________________________________ conv5_block10_0_relu (Activatio (None, None, None, 8 0 conv5_block10_0_bn[0][0] __________________________________________________________________________________________________ conv5_block10_1_conv (Conv2D) (None, None, None, 1 102400 conv5_block10_0_relu[0][0] __________________________________________________________________________________________________ conv5_block10_1_bn (BatchNormal (None, None, None, 1 512 conv5_block10_1_conv[0][0] __________________________________________________________________________________________________ conv5_block10_1_relu (Activatio (None, None, None, 1 0 conv5_block10_1_bn[0][0] __________________________________________________________________________________________________ conv5_block10_2_conv (Conv2D) (None, None, None, 3 36864 conv5_block10_1_relu[0][0] __________________________________________________________________________________________________ conv5_block10_concat (Concatena (None, None, None, 8 0 conv5_block9_concat[0][0] conv5_block10_2_conv[0][0] __________________________________________________________________________________________________ conv5_block11_0_bn (BatchNormal (None, None, None, 8 3328 conv5_block10_concat[0][0] __________________________________________________________________________________________________ conv5_block11_0_relu (Activatio (None, None, None, 8 0 conv5_block11_0_bn[0][0] __________________________________________________________________________________________________ conv5_block11_1_conv (Conv2D) (None, None, None, 1 106496 conv5_block11_0_relu[0][0] __________________________________________________________________________________________________ conv5_block11_1_bn (BatchNormal (None, None, None, 1 512 conv5_block11_1_conv[0][0] __________________________________________________________________________________________________ conv5_block11_1_relu (Activatio (None, None, None, 1 0 conv5_block11_1_bn[0][0] __________________________________________________________________________________________________ conv5_block11_2_conv (Conv2D) (None, None, None, 3 36864 conv5_block11_1_relu[0][0] __________________________________________________________________________________________________ conv5_block11_concat (Concatena (None, None, None, 8 0 conv5_block10_concat[0][0] conv5_block11_2_conv[0][0] __________________________________________________________________________________________________ conv5_block12_0_bn (BatchNormal (None, None, None, 8 3456 conv5_block11_concat[0][0] __________________________________________________________________________________________________ conv5_block12_0_relu (Activatio (None, None, None, 8 0 conv5_block12_0_bn[0][0] __________________________________________________________________________________________________ conv5_block12_1_conv (Conv2D) (None, None, None, 1 110592 conv5_block12_0_relu[0][0] __________________________________________________________________________________________________ conv5_block12_1_bn (BatchNormal (None, None, None, 1 512 conv5_block12_1_conv[0][0] __________________________________________________________________________________________________ conv5_block12_1_relu (Activatio (None, None, None, 1 0 conv5_block12_1_bn[0][0] __________________________________________________________________________________________________ conv5_block12_2_conv (Conv2D) (None, None, None, 3 36864 conv5_block12_1_relu[0][0] __________________________________________________________________________________________________ conv5_block12_concat (Concatena (None, None, None, 8 0 conv5_block11_concat[0][0] conv5_block12_2_conv[0][0] __________________________________________________________________________________________________ conv5_block13_0_bn (BatchNormal (None, None, None, 8 3584 conv5_block12_concat[0][0] __________________________________________________________________________________________________ conv5_block13_0_relu (Activatio (None, None, None, 8 0 conv5_block13_0_bn[0][0] __________________________________________________________________________________________________ conv5_block13_1_conv (Conv2D) (None, None, None, 1 114688 conv5_block13_0_relu[0][0] __________________________________________________________________________________________________ conv5_block13_1_bn (BatchNormal (None, None, None, 1 512 conv5_block13_1_conv[0][0] __________________________________________________________________________________________________ conv5_block13_1_relu (Activatio (None, None, None, 1 0 conv5_block13_1_bn[0][0] __________________________________________________________________________________________________ conv5_block13_2_conv (Conv2D) (None, None, None, 3 36864 conv5_block13_1_relu[0][0] __________________________________________________________________________________________________ conv5_block13_concat (Concatena (None, None, None, 9 0 conv5_block12_concat[0][0] conv5_block13_2_conv[0][0] __________________________________________________________________________________________________ conv5_block14_0_bn (BatchNormal (None, None, None, 9 3712 conv5_block13_concat[0][0] __________________________________________________________________________________________________ conv5_block14_0_relu (Activatio (None, None, None, 9 0 conv5_block14_0_bn[0][0] __________________________________________________________________________________________________ conv5_block14_1_conv (Conv2D) (None, None, None, 1 118784 conv5_block14_0_relu[0][0] __________________________________________________________________________________________________ conv5_block14_1_bn (BatchNormal (None, None, None, 1 512 conv5_block14_1_conv[0][0] __________________________________________________________________________________________________ conv5_block14_1_relu (Activatio (None, None, None, 1 0 conv5_block14_1_bn[0][0] __________________________________________________________________________________________________ conv5_block14_2_conv (Conv2D) (None, None, None, 3 36864 conv5_block14_1_relu[0][0] __________________________________________________________________________________________________ conv5_block14_concat (Concatena (None, None, None, 9 0 conv5_block13_concat[0][0] conv5_block14_2_conv[0][0] __________________________________________________________________________________________________ conv5_block15_0_bn (BatchNormal (None, None, None, 9 3840 conv5_block14_concat[0][0] __________________________________________________________________________________________________ conv5_block15_0_relu (Activatio (None, None, None, 9 0 conv5_block15_0_bn[0][0] __________________________________________________________________________________________________ conv5_block15_1_conv (Conv2D) (None, None, None, 1 122880 conv5_block15_0_relu[0][0] __________________________________________________________________________________________________ conv5_block15_1_bn (BatchNormal (None, None, None, 1 512 conv5_block15_1_conv[0][0] __________________________________________________________________________________________________ conv5_block15_1_relu (Activatio (None, None, None, 1 0 conv5_block15_1_bn[0][0] __________________________________________________________________________________________________ conv5_block15_2_conv (Conv2D) (None, None, None, 3 36864 conv5_block15_1_relu[0][0] __________________________________________________________________________________________________ conv5_block15_concat (Concatena (None, None, None, 9 0 conv5_block14_concat[0][0] conv5_block15_2_conv[0][0] __________________________________________________________________________________________________ conv5_block16_0_bn (BatchNormal (None, None, None, 9 3968 conv5_block15_concat[0][0] __________________________________________________________________________________________________ conv5_block16_0_relu (Activatio (None, None, None, 9 0 conv5_block16_0_bn[0][0] __________________________________________________________________________________________________ conv5_block16_1_conv (Conv2D) (None, None, None, 1 126976 conv5_block16_0_relu[0][0] __________________________________________________________________________________________________ conv5_block16_1_bn (BatchNormal (None, None, None, 1 512 conv5_block16_1_conv[0][0] __________________________________________________________________________________________________ conv5_block16_1_relu (Activatio (None, None, None, 1 0 conv5_block16_1_bn[0][0] __________________________________________________________________________________________________ conv5_block16_2_conv (Conv2D) (None, None, None, 3 36864 conv5_block16_1_relu[0][0] __________________________________________________________________________________________________ conv5_block16_concat (Concatena (None, None, None, 1 0 conv5_block15_concat[0][0] conv5_block16_2_conv[0][0] __________________________________________________________________________________________________ bn (BatchNormalization) (None, None, None, 1 4096 conv5_block16_concat[0][0] __________________________________________________________________________________________________ global_average_pooling2d_1 (Glo (None, 1024) 0 bn[0][0] __________________________________________________________________________________________________ dense_1 (Dense) (None, 14) 14350 global_average_pooling2d_1[0][0] ================================================================================================== Total params: 7,051,854 Trainable params: 6,968,206 Non-trainable params: 83,648 __________________________________________________________________________________________________
Keras models include abundant information about the elements that make them up. You can check all of the available methods and attributes of this class by using the dir()
method:
# Printing out methods and attributes for Keras model
print(f"Keras' models have the following methods and attributes: \n\n{dir(model)}")
Keras' models have the following methods and attributes: ['__call__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_add_inbound_node', '_built', '_check_num_samples', '_check_trainable_weights_consistency', '_collected_trainable_weights', '_container_nodes', '_feed_input_names', '_feed_input_shapes', '_feed_inputs', '_feed_loss_fns', '_feed_output_names', '_feed_output_shapes', '_feed_outputs', '_feed_sample_weight_modes', '_feed_sample_weights', '_feed_targets', '_fit_loop', '_function_kwargs', '_get_node_attribute_at_index', '_inbound_nodes', '_internal_input_shapes', '_internal_output_shapes', '_make_predict_function', '_make_test_function', '_make_train_function', '_node_key', '_nodes_by_depth', '_outbound_nodes', '_output_mask_cache', '_output_shape_cache', '_output_tensor_cache', '_per_input_losses', '_per_input_updates', '_predict_loop', '_standardize_user_data', '_test_loop', '_updated_config', 'add_loss', 'add_update', 'add_weight', 'assert_input_compatibility', 'build', 'built', 'call', 'compile', 'compute_mask', 'compute_output_shape', 'count_params', 'evaluate', 'evaluate_generator', 'fit', 'fit_generator', 'from_config', 'get_config', 'get_input_at', 'get_input_mask_at', 'get_input_shape_at', 'get_layer', 'get_losses_for', 'get_output_at', 'get_output_mask_at', 'get_output_shape_at', 'get_updates_for', 'get_weights', 'input', 'input_layers', 'input_layers_node_indices', 'input_layers_tensor_indices', 'input_mask', 'input_names', 'input_shape', 'input_spec', 'inputs', 'layers', 'layers_by_depth', 'load_weights', 'loss', 'loss_functions', 'loss_weights', 'losses', 'metrics', 'metrics_names', 'metrics_tensors', 'metrics_updates', 'name', 'non_trainable_weights', 'optimizer', 'output', 'output_layers', 'output_layers_node_indices', 'output_layers_tensor_indices', 'output_mask', 'output_names', 'output_shape', 'outputs', 'predict', 'predict_function', 'predict_generator', 'predict_on_batch', 'reset_states', 'run_internal_graph', 'sample_weight_mode', 'sample_weight_modes', 'sample_weights', 'save', 'save_weights', 'set_weights', 'state_updates', 'stateful', 'stateful_metric_functions', 'stateful_metric_names', 'summary', 'supports_masking', 'targets', 'test_function', 'test_on_batch', 'to_json', 'to_yaml', 'total_loss', 'train_function', 'train_on_batch', 'trainable', 'trainable_weights', 'updates', 'uses_learning_phase', 'weighted_metrics', 'weights']
Wow, this certainly is a lot! These models are indeed very complex.
What we are interested in are the layers of the model which can be easily accessed as an attribute using the dot notation. They are a list of layers, which can be confirmed by checking its type:
# Check the type of the model's layers
type(model.layers)
# Print 5 first layers along with their names
for i in range(5):
l = model.layers[i]
print(f"Layer number {i}: \n{l} \nWith name: {l.name} \n")
Let's check how many layers our model has:
# Print number of layers in our model
print(f"The model has {len(model.layers)} layers")
Our main goal is interpreting the representations which the neural net is creating for classifying our images. But as you can see this architecture has many layers.
Actually we are really interested in the representations that the convolutional layers produce because these are the layers that (hopefully) recognize concrete elements within the images. We are also interested in the "concatenate" layers because in our model's arquitecture they concatenate convolutional layers.
Let's check how many of those we have:
# Number of layers that are of type "Convolutional" or "Concatenate"
len([l for l in model.layers if ("conv" in str(type(l))) or ("Concatenate" in str(type(l)))])
This number is still very big to try to interpret each one of these layers individually.
One characteristic of CNN's is that the earlier layers capture low-level features such as edges in an image while the deeper layers capture high-level concepts such as physical features of a "Cat".
Because of this Grad-CAM usually focuses on the last layers, as they provide a better picture of what the network is paying attention to when classifying a particular class. Let's grab the last concatenate layer of our model. Luckily Keras API makes this quite easy:
# Save the desired layer in a variable
layer = model.layers[424]
# Print layer
layer
This approach is not the best since we will need to know the exact index of the desired layer. Luckily we can use the get_layer()
method in conjunction with the layer's name to get the same result.
Remember you can get the name from the information displayed earlier with the summary()
method.
# Save the desired layer in a variable
layer = model.get_layer("conv5_block16_concat")
# Print layer
layer
Let's check what methods and attributes we have available when working with this layer:
# Printing out methods and attributes for Keras' layer
print(f"Keras' layers have the following methods and attributes: \n\n{dir(layer)}")
Since we want to know the representations which this layer is abstracting from the images we should be interested in the output from this layer. Luckily we have this attribute available:
# Print layer's output
layer.output
Do you notice something odd? The shape of this tensor is undefined for some dimensions. This is because this tensor is just a placeholder and it doesn't really contain information about the activations that occurred in this layer.
To compute the actual activation values given an input we will need to use a Keras function.
This function accepts lists of input and output placeholders and can be used with an actual input to compute the respective output of the layer associated to the placeholder for that given input.
Before jumping onto the Keras function we should rewind a little bit to get the placeholder tensor associated with the input. You can get this from the model’s input:
# Print model's input tensor placeholder
model.input
We can see that this is a placeholder as well. Now let's instantiate our Keras function using Keras backend. Please be aware that this function expects its arguments as lists or tuples:
# Instantiate the function to compute the activations of the last convolutional layer
last_layer_activations_function = K.function([model.input], [layer.output])
# Print the Keras function
last_layer_activations_function
Let's test the functions for computing the last layer activation which we just defined on a particular image. Don't worry about the code to load the image, this has been taken care of for you. You should only care that an image ready to be processed will be saved in the x variable:
# Load dataframe that contains information about the dataset of images
df = pd.read_csv("nih_new/train-small.csv")
# Path to the actual image
im_path = 'nih_new/images-small/00000599_000.png'
# Load the image and save it to a variable
x = load_image(im_path, df, preprocess=False)
# Display the image
plt.imshow(x, cmap = 'gray')
plt.show()
We should normalize this image before going forward, this has also been taken care of:
# Calculate mean and standard deviation of a batch of images
mean, std = get_mean_std_per_batch(df)
# Normalize image
x = load_image_normalize(im_path, mean, std)
Now we have everything we need to compute the actual values of the last layer activations. In this case we should also provide the input as a list or tuple:
# Run the function on the image and save it in a variable
actual_activations = last_layer_activations_function([x])
An important intermediary step is to trim the batch dimension which can be done like this. This is necessary because we are applying Grad-CAM to a single image rather than to a batch of images:
# Remove batch dimension
actual_activations = actual_activations[0][0, :]
# Print shape of the activation array
print(f"Activations of last convolutional layer have shape: {actual_activations.shape}")
# Print activation array
actual_activations
Looks like everything worked out nicely! This is all for this lecture notebook (Grad-CAM Part 1). In Part 2 we will see how to calculate the gradients of the model's output with respect to the activations in this layer. This is the "Grad" part of Grad-CAM.
Congratulations on finishing this lecture notebook! Hopefully you will now have a better understanding of how to leverage Keras's API power for computing activations in specific layers. Keep it up!