Deep Learning Models -- A collection of various deep learning architectures, models, and tips for TensorFlow and PyTorch in Jupyter Notebooks.

In [1]:
%load_ext watermark
%watermark -a 'Sebastian Raschka' -v -p torch
Sebastian Raschka 

CPython 3.6.8
IPython 7.2.0

torch 1.1.0
  • Runs on CPU or GPU (if available)

Model Zoo -- Ordinal Regression CNN -- CORAL

Implementation of a method for ordinal regression, CORAL [1] (COnsistent RAnk Logits) applied to predicting age from face images in the AFAD [2] (Asian Face) dataset using a simple ResNet-34 [3] convolutional network architecture.

Note that in order to reduce training time, only a subset of AFAD (AFAD-Lite) is being used.

Imports

In [2]:
import time
import numpy as np
import pandas as pd
import os

import torch.nn as nn
import torch.nn.functional as F
import torch

from torch.utils.data import Dataset
from torch.utils.data import DataLoader
from torchvision import transforms
from PIL import Image


if torch.cuda.is_available():
    torch.backends.cudnn.deterministic = True

Downloading the Dataset

In [3]:
!git clone https://github.com/afad-dataset/tarball-lite.git
Cloning into 'tarball-lite'...
remote: Enumerating objects: 37, done.
remote: Total 37 (delta 0), reused 0 (delta 0), pack-reused 37
Unpacking objects: 100% (37/37), done.
Checking out files: 100% (30/30), done.
In [4]:
!cat tarball-lite/AFAD-Lite.tar.xz* > tarball-lite/AFAD-Lite.tar.xz
In [5]:
!tar xf tarball-lite/AFAD-Lite.tar.xz
In [6]:
rootDir = 'AFAD-Lite'

files = [os.path.relpath(os.path.join(dirpath, file), rootDir)
         for (dirpath, dirnames, filenames) in os.walk(rootDir) 
         for file in filenames if file.endswith('.jpg')]
In [7]:
len(files)
Out[7]:
59344
In [8]:
d = {}

d['age'] = []
d['gender'] = []
d['file'] = []
d['path'] = []

for f in files:
    age, gender, fname = f.split('/')
    if gender == '111':
        gender = 'male'
    else:
        gender = 'female'
        
    d['age'].append(age)
    d['gender'].append(gender)
    d['file'].append(fname)
    d['path'].append(f)
In [9]:
df = pd.DataFrame.from_dict(d)
df.head()
Out[9]:
age gender file path
0 39 female 474596-0.jpg 39/112/474596-0.jpg
1 39 female 397477-0.jpg 39/112/397477-0.jpg
2 39 female 576466-0.jpg 39/112/576466-0.jpg
3 39 female 399405-0.jpg 39/112/399405-0.jpg
4 39 female 410524-0.jpg 39/112/410524-0.jpg
In [10]:
df['age'].min()
Out[10]:
'18'
In [11]:
df['age'] = df['age'].values.astype(int) - 18
In [12]:
np.random.seed(123)
msk = np.random.rand(len(df)) < 0.8
df_train = df[msk]
df_test = df[~msk]
In [13]:
df_train.set_index('file', inplace=True)
df_train.to_csv('training_set_lite.csv')
In [14]:
df_test.set_index('file', inplace=True)
df_test.to_csv('test_set_lite.csv')
In [15]:
num_ages = np.unique(df['age'].values).shape[0]
print(num_ages)
22

Settings

In [16]:
##########################
### SETTINGS
##########################

# Device
DEVICE = torch.device("cuda:3" if torch.cuda.is_available() else "cpu")

NUM_WORKERS = 4

NUM_CLASSES = num_ages
BATCH_SIZE = 512
NUM_EPOCHS = 150
LEARNING_RATE = 0.0005
RANDOM_SEED = 123
GRAYSCALE = False

TRAIN_CSV_PATH = 'training_set_lite.csv'
TEST_CSV_PATH = 'test_set_lite.csv'
IMAGE_PATH = 'AFAD-Lite'

Dataset Loaders

In [17]:
class AFADDatasetAge(Dataset):
    """Custom Dataset for loading AFAD face images"""

    def __init__(self, csv_path, img_dir, transform=None):

        df = pd.read_csv(csv_path, index_col=0)
        self.img_dir = img_dir
        self.csv_path = csv_path
        self.img_paths = df['path']
        self.y = df['age'].values
        self.transform = transform

    def __getitem__(self, index):
        img = Image.open(os.path.join(self.img_dir,
                                      self.img_paths[index]))

        if self.transform is not None:
            img = self.transform(img)

        label = self.y[index]
        levels = [1]*label + [0]*(NUM_CLASSES - 1 - label)
        levels = torch.tensor(levels, dtype=torch.float32)

        return img, label, levels

    def __len__(self):
        return self.y.shape[0]


custom_transform = transforms.Compose([transforms.Resize((128, 128)),
                                       transforms.RandomCrop((120, 120)),
                                       transforms.ToTensor()])

train_dataset = AFADDatasetAge(csv_path=TRAIN_CSV_PATH,
                               img_dir=IMAGE_PATH,
                               transform=custom_transform)


custom_transform2 = transforms.Compose([transforms.Resize((128, 128)),
                                        transforms.CenterCrop((120, 120)),
                                        transforms.ToTensor()])

test_dataset = AFADDatasetAge(csv_path=TEST_CSV_PATH,
                              img_dir=IMAGE_PATH,
                              transform=custom_transform2)


train_loader = DataLoader(dataset=train_dataset,
                          batch_size=BATCH_SIZE,
                          shuffle=True,
                          num_workers=NUM_WORKERS)

test_loader = DataLoader(dataset=test_dataset,
                         batch_size=BATCH_SIZE,
                         shuffle=False,
                         num_workers=NUM_WORKERS)

Model

In [18]:
##########################
# MODEL
##########################


def conv3x3(in_planes, out_planes, stride=1):
    """3x3 convolution with padding"""
    return nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=stride,
                     padding=1, bias=False)


class BasicBlock(nn.Module):
    expansion = 1

    def __init__(self, inplanes, planes, stride=1, downsample=None):
        super(BasicBlock, self).__init__()
        self.conv1 = conv3x3(inplanes, planes, stride)
        self.bn1 = nn.BatchNorm2d(planes)
        self.relu = nn.ReLU(inplace=True)
        self.conv2 = conv3x3(planes, planes)
        self.bn2 = nn.BatchNorm2d(planes)
        self.downsample = downsample
        self.stride = stride

    def forward(self, x):
        residual = x

        out = self.conv1(x)
        out = self.bn1(out)
        out = self.relu(out)

        out = self.conv2(out)
        out = self.bn2(out)

        if self.downsample is not None:
            residual = self.downsample(x)

        out += residual
        out = self.relu(out)

        return out


class ResNet(nn.Module):

    def __init__(self, block, layers, num_classes, grayscale):
        self.num_classes = num_classes
        self.inplanes = 64
        if grayscale:
            in_dim = 1
        else:
            in_dim = 3
        super(ResNet, self).__init__()
        self.conv1 = nn.Conv2d(in_dim, 64, kernel_size=7, stride=2, padding=3,
                               bias=False)
        self.bn1 = nn.BatchNorm2d(64)
        self.relu = nn.ReLU(inplace=True)
        self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
        self.layer1 = self._make_layer(block, 64, layers[0])
        self.layer2 = self._make_layer(block, 128, layers[1], stride=2)
        self.layer3 = self._make_layer(block, 256, layers[2], stride=2)
        self.layer4 = self._make_layer(block, 512, layers[3], stride=2)
        self.avgpool = nn.AvgPool2d(7, stride=1, padding=2)
        self.fc = nn.Linear(2048 * block.expansion, 1, bias=False)
        self.linear_1_bias = nn.Parameter(torch.zeros(self.num_classes-1).float())

        for m in self.modules():
            if isinstance(m, nn.Conv2d):
                n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
                m.weight.data.normal_(0, (2. / n)**.5)
            elif isinstance(m, nn.BatchNorm2d):
                m.weight.data.fill_(1)
                m.bias.data.zero_()

    def _make_layer(self, block, planes, blocks, stride=1):
        downsample = None
        if stride != 1 or self.inplanes != planes * block.expansion:
            downsample = nn.Sequential(
                nn.Conv2d(self.inplanes, planes * block.expansion,
                          kernel_size=1, stride=stride, bias=False),
                nn.BatchNorm2d(planes * block.expansion),
            )

        layers = []
        layers.append(block(self.inplanes, planes, stride, downsample))
        self.inplanes = planes * block.expansion
        for i in range(1, blocks):
            layers.append(block(self.inplanes, planes))

        return nn.Sequential(*layers)

    def forward(self, x):
        x = self.conv1(x)
        x = self.bn1(x)
        x = self.relu(x)
        x = self.maxpool(x)

        x = self.layer1(x)
        x = self.layer2(x)
        x = self.layer3(x)
        x = self.layer4(x)

        x = self.avgpool(x)
        x = x.view(x.size(0), -1)
        logits = self.fc(x)
        logits = logits + self.linear_1_bias
        probas = torch.sigmoid(logits)
        return logits, probas


def resnet34(num_classes, grayscale):
    """Constructs a ResNet-34 model."""
    model = ResNet(block=BasicBlock,
                   layers=[3, 4, 6, 3],
                   num_classes=num_classes,
                   grayscale=grayscale)
    return model
In [19]:
###########################################
# Initialize Cost, Model, and Optimizer
###########################################

def cost_fn(logits, levels):
    val = (-torch.sum((F.logsigmoid(logits)*levels
                      + (F.logsigmoid(logits) - logits)*(1-levels)),
           dim=1))
    return torch.mean(val)


torch.manual_seed(RANDOM_SEED)
torch.cuda.manual_seed(RANDOM_SEED)
model = resnet34(NUM_CLASSES, GRAYSCALE)

model.to(DEVICE)
optimizer = torch.optim.Adam(model.parameters(), lr=LEARNING_RATE)

Training

In [20]:
def compute_mae_and_mse(model, data_loader, device):
    mae, mse, num_examples = 0, 0, 0
    for i, (features, targets, levels) in enumerate(data_loader):

        features = features.to(device)
        targets = targets.to(device)

        logits, probas = model(features)
        predict_levels = probas > 0.5
        predicted_labels = torch.sum(predict_levels, dim=1)
        num_examples += targets.size(0)
        mae += torch.sum(torch.abs(predicted_labels - targets))
        mse += torch.sum((predicted_labels - targets)**2)
    mae = mae.float() / num_examples
    mse = mse.float() / num_examples
    return mae, mse


start_time = time.time()
for epoch in range(NUM_EPOCHS):

    model.train()
    for batch_idx, (features, targets, levels) in enumerate(train_loader):

        features = features.to(DEVICE)
        targets = targets
        targets = targets.to(DEVICE)
        levels = levels.to(DEVICE)

        # FORWARD AND BACK PROP
        logits, probas = model(features)
        cost = cost_fn(logits, levels)
        optimizer.zero_grad()

        cost.backward()

        # UPDATE MODEL PARAMETERS
        optimizer.step()

        # LOGGING
        if not batch_idx % 150:
            s = ('Epoch: %03d/%03d | Batch %04d/%04d | Cost: %.4f'
                 % (epoch+1, NUM_EPOCHS, batch_idx,
                     len(train_dataset)//BATCH_SIZE, cost))
            print(s)

    s = 'Time elapsed: %.2f min' % ((time.time() - start_time)/60)
    print(s)
Epoch: 001/150 | Batch 0000/0092 | Cost: 15.0424
Time elapsed: 0.91 min
Epoch: 002/150 | Batch 0000/0092 | Cost: 12.5222
Time elapsed: 1.83 min
Epoch: 003/150 | Batch 0000/0092 | Cost: 12.0170
Time elapsed: 2.77 min
Epoch: 004/150 | Batch 0000/0092 | Cost: 11.6722
Time elapsed: 3.71 min
Epoch: 005/150 | Batch 0000/0092 | Cost: 11.2609
Time elapsed: 4.65 min
Epoch: 006/150 | Batch 0000/0092 | Cost: 10.9205
Time elapsed: 5.59 min
Epoch: 007/150 | Batch 0000/0092 | Cost: 11.2049
Time elapsed: 6.54 min
Epoch: 008/150 | Batch 0000/0092 | Cost: 10.4912
Time elapsed: 7.50 min
Epoch: 009/150 | Batch 0000/0092 | Cost: 10.2098
Time elapsed: 8.46 min
Epoch: 010/150 | Batch 0000/0092 | Cost: 10.0003
Time elapsed: 9.41 min
Epoch: 011/150 | Batch 0000/0092 | Cost: 9.9253
Time elapsed: 10.36 min
Epoch: 012/150 | Batch 0000/0092 | Cost: 9.5460
Time elapsed: 11.31 min
Epoch: 013/150 | Batch 0000/0092 | Cost: 9.3959
Time elapsed: 12.26 min
Epoch: 014/150 | Batch 0000/0092 | Cost: 9.2571
Time elapsed: 13.21 min
Epoch: 015/150 | Batch 0000/0092 | Cost: 8.9584
Time elapsed: 14.16 min
Epoch: 016/150 | Batch 0000/0092 | Cost: 8.7167
Time elapsed: 15.13 min
Epoch: 017/150 | Batch 0000/0092 | Cost: 8.5358
Time elapsed: 16.09 min
Epoch: 018/150 | Batch 0000/0092 | Cost: 8.2750
Time elapsed: 17.05 min
Epoch: 019/150 | Batch 0000/0092 | Cost: 8.0949
Time elapsed: 18.02 min
Epoch: 020/150 | Batch 0000/0092 | Cost: 7.8891
Time elapsed: 19.00 min
Epoch: 021/150 | Batch 0000/0092 | Cost: 7.7796
Time elapsed: 19.96 min
Epoch: 022/150 | Batch 0000/0092 | Cost: 7.5751
Time elapsed: 20.92 min
Epoch: 023/150 | Batch 0000/0092 | Cost: 7.4735
Time elapsed: 21.88 min
Epoch: 024/150 | Batch 0000/0092 | Cost: 7.3455
Time elapsed: 22.84 min
Epoch: 025/150 | Batch 0000/0092 | Cost: 7.1206
Time elapsed: 23.80 min
Epoch: 026/150 | Batch 0000/0092 | Cost: 7.0157
Time elapsed: 24.75 min
Epoch: 027/150 | Batch 0000/0092 | Cost: 6.8013
Time elapsed: 25.71 min
Epoch: 028/150 | Batch 0000/0092 | Cost: 6.6981
Time elapsed: 26.67 min
Epoch: 029/150 | Batch 0000/0092 | Cost: 6.6510
Time elapsed: 27.62 min
Epoch: 030/150 | Batch 0000/0092 | Cost: 6.4859
Time elapsed: 28.58 min
Epoch: 031/150 | Batch 0000/0092 | Cost: 6.3101
Time elapsed: 29.54 min
Epoch: 032/150 | Batch 0000/0092 | Cost: 6.2179
Time elapsed: 30.49 min
Epoch: 033/150 | Batch 0000/0092 | Cost: 6.2418
Time elapsed: 31.45 min
Epoch: 034/150 | Batch 0000/0092 | Cost: 6.0992
Time elapsed: 32.41 min
Epoch: 035/150 | Batch 0000/0092 | Cost: 5.9214
Time elapsed: 33.37 min
Epoch: 036/150 | Batch 0000/0092 | Cost: 5.8149
Time elapsed: 34.33 min
Epoch: 037/150 | Batch 0000/0092 | Cost: 5.7312
Time elapsed: 35.30 min
Epoch: 038/150 | Batch 0000/0092 | Cost: 5.6387
Time elapsed: 36.28 min
Epoch: 039/150 | Batch 0000/0092 | Cost: 5.5805
Time elapsed: 37.24 min
Epoch: 040/150 | Batch 0000/0092 | Cost: 5.3195
Time elapsed: 38.20 min
Epoch: 041/150 | Batch 0000/0092 | Cost: 5.5065
Time elapsed: 39.16 min
Epoch: 042/150 | Batch 0000/0092 | Cost: 5.6153
Time elapsed: 40.13 min
Epoch: 043/150 | Batch 0000/0092 | Cost: 5.2801
Time elapsed: 41.10 min
Epoch: 044/150 | Batch 0000/0092 | Cost: 5.2717
Time elapsed: 42.07 min
Epoch: 045/150 | Batch 0000/0092 | Cost: 5.1263
Time elapsed: 43.06 min
Epoch: 046/150 | Batch 0000/0092 | Cost: 5.0700
Time elapsed: 44.03 min
Epoch: 047/150 | Batch 0000/0092 | Cost: 5.1728
Time elapsed: 45.01 min
Epoch: 048/150 | Batch 0000/0092 | Cost: 5.0284
Time elapsed: 45.98 min
Epoch: 049/150 | Batch 0000/0092 | Cost: 4.9178
Time elapsed: 46.95 min
Epoch: 050/150 | Batch 0000/0092 | Cost: 5.0401
Time elapsed: 47.93 min
Epoch: 051/150 | Batch 0000/0092 | Cost: 4.7706
Time elapsed: 48.92 min
Epoch: 052/150 | Batch 0000/0092 | Cost: 4.8608
Time elapsed: 49.90 min
Epoch: 053/150 | Batch 0000/0092 | Cost: 4.7105
Time elapsed: 50.87 min
Epoch: 054/150 | Batch 0000/0092 | Cost: 4.7156
Time elapsed: 51.85 min
Epoch: 055/150 | Batch 0000/0092 | Cost: 4.6754
Time elapsed: 52.81 min
Epoch: 056/150 | Batch 0000/0092 | Cost: 4.5800
Time elapsed: 53.79 min
Epoch: 057/150 | Batch 0000/0092 | Cost: 4.4490
Time elapsed: 54.76 min
Epoch: 058/150 | Batch 0000/0092 | Cost: 4.4306
Time elapsed: 55.74 min
Epoch: 059/150 | Batch 0000/0092 | Cost: 4.4310
Time elapsed: 56.70 min
Epoch: 060/150 | Batch 0000/0092 | Cost: 4.4331
Time elapsed: 57.67 min
Epoch: 061/150 | Batch 0000/0092 | Cost: 4.2809
Time elapsed: 58.64 min
Epoch: 062/150 | Batch 0000/0092 | Cost: 4.3698
Time elapsed: 59.62 min
Epoch: 063/150 | Batch 0000/0092 | Cost: 4.3086
Time elapsed: 60.59 min
Epoch: 064/150 | Batch 0000/0092 | Cost: 4.2474
Time elapsed: 61.57 min
Epoch: 065/150 | Batch 0000/0092 | Cost: 4.2255
Time elapsed: 62.53 min
Epoch: 066/150 | Batch 0000/0092 | Cost: 4.1545
Time elapsed: 63.52 min
Epoch: 067/150 | Batch 0000/0092 | Cost: 4.1680
Time elapsed: 64.49 min
Epoch: 068/150 | Batch 0000/0092 | Cost: 4.1133
Time elapsed: 65.46 min
Epoch: 069/150 | Batch 0000/0092 | Cost: 4.0342
Time elapsed: 66.42 min
Epoch: 070/150 | Batch 0000/0092 | Cost: 4.1035
Time elapsed: 67.38 min
Epoch: 071/150 | Batch 0000/0092 | Cost: 4.0500
Time elapsed: 68.34 min
Epoch: 072/150 | Batch 0000/0092 | Cost: 3.8781
Time elapsed: 69.31 min
Epoch: 073/150 | Batch 0000/0092 | Cost: 3.8854
Time elapsed: 70.29 min
Epoch: 074/150 | Batch 0000/0092 | Cost: 3.9859
Time elapsed: 71.25 min
Epoch: 075/150 | Batch 0000/0092 | Cost: 4.0262
Time elapsed: 72.22 min
Epoch: 076/150 | Batch 0000/0092 | Cost: 4.3140
Time elapsed: 73.21 min
Epoch: 077/150 | Batch 0000/0092 | Cost: 4.1002
Time elapsed: 74.20 min
Epoch: 078/150 | Batch 0000/0092 | Cost: 3.9676
Time elapsed: 75.19 min
Epoch: 079/150 | Batch 0000/0092 | Cost: 3.6617
Time elapsed: 76.18 min
Epoch: 080/150 | Batch 0000/0092 | Cost: 3.7342
Time elapsed: 77.15 min
Epoch: 081/150 | Batch 0000/0092 | Cost: 3.5710
Time elapsed: 78.12 min
Epoch: 082/150 | Batch 0000/0092 | Cost: 3.6218
Time elapsed: 79.08 min
Epoch: 083/150 | Batch 0000/0092 | Cost: 3.4883
Time elapsed: 80.04 min
Epoch: 084/150 | Batch 0000/0092 | Cost: 3.5037
Time elapsed: 81.01 min
Epoch: 085/150 | Batch 0000/0092 | Cost: 3.4316
Time elapsed: 81.97 min
Epoch: 086/150 | Batch 0000/0092 | Cost: 3.4448
Time elapsed: 82.94 min
Epoch: 087/150 | Batch 0000/0092 | Cost: 3.3413
Time elapsed: 83.89 min
Epoch: 088/150 | Batch 0000/0092 | Cost: 3.4418
Time elapsed: 84.86 min
Epoch: 089/150 | Batch 0000/0092 | Cost: 3.4258
Time elapsed: 85.82 min
Epoch: 090/150 | Batch 0000/0092 | Cost: 3.3049
Time elapsed: 86.78 min
Epoch: 091/150 | Batch 0000/0092 | Cost: 3.2554
Time elapsed: 87.73 min
Epoch: 092/150 | Batch 0000/0092 | Cost: 3.2919
Time elapsed: 88.69 min
Epoch: 093/150 | Batch 0000/0092 | Cost: 3.3172
Time elapsed: 89.65 min
Epoch: 094/150 | Batch 0000/0092 | Cost: 3.5744
Time elapsed: 90.62 min
Epoch: 095/150 | Batch 0000/0092 | Cost: 4.5396
Time elapsed: 91.58 min
Epoch: 096/150 | Batch 0000/0092 | Cost: 3.7548
Time elapsed: 92.54 min
Epoch: 097/150 | Batch 0000/0092 | Cost: 3.4449
Time elapsed: 93.49 min
Epoch: 098/150 | Batch 0000/0092 | Cost: 3.3186
Time elapsed: 94.46 min
Epoch: 099/150 | Batch 0000/0092 | Cost: 3.2050
Time elapsed: 95.42 min
Epoch: 100/150 | Batch 0000/0092 | Cost: 3.1218
Time elapsed: 96.38 min
Epoch: 101/150 | Batch 0000/0092 | Cost: 3.0612
Time elapsed: 97.35 min
Epoch: 102/150 | Batch 0000/0092 | Cost: 3.0640
Time elapsed: 98.31 min
Epoch: 103/150 | Batch 0000/0092 | Cost: 2.8820
Time elapsed: 99.27 min
Epoch: 104/150 | Batch 0000/0092 | Cost: 2.9511
Time elapsed: 100.23 min
Epoch: 105/150 | Batch 0000/0092 | Cost: 2.9219
Time elapsed: 101.19 min
Epoch: 106/150 | Batch 0000/0092 | Cost: 2.9429
Time elapsed: 102.15 min
Epoch: 107/150 | Batch 0000/0092 | Cost: 2.8934
Time elapsed: 103.11 min
Epoch: 108/150 | Batch 0000/0092 | Cost: 2.8541
Time elapsed: 104.06 min
Epoch: 109/150 | Batch 0000/0092 | Cost: 2.8962
Time elapsed: 105.03 min
Epoch: 110/150 | Batch 0000/0092 | Cost: 2.8225
Time elapsed: 105.99 min
Epoch: 111/150 | Batch 0000/0092 | Cost: 2.7968
Time elapsed: 106.96 min
Epoch: 112/150 | Batch 0000/0092 | Cost: 2.7319
Time elapsed: 107.93 min
Epoch: 113/150 | Batch 0000/0092 | Cost: 2.6711
Time elapsed: 108.89 min
Epoch: 114/150 | Batch 0000/0092 | Cost: 2.8028
Time elapsed: 109.86 min
Epoch: 115/150 | Batch 0000/0092 | Cost: 2.7948
Time elapsed: 110.83 min
Epoch: 116/150 | Batch 0000/0092 | Cost: 2.7675
Time elapsed: 111.79 min
Epoch: 117/150 | Batch 0000/0092 | Cost: 2.8945
Time elapsed: 112.75 min
Epoch: 118/150 | Batch 0000/0092 | Cost: 4.3488
Time elapsed: 113.71 min
Epoch: 119/150 | Batch 0000/0092 | Cost: 3.8014
Time elapsed: 114.67 min
Epoch: 120/150 | Batch 0000/0092 | Cost: 3.3284
Time elapsed: 115.64 min
Epoch: 121/150 | Batch 0000/0092 | Cost: 2.9553
Time elapsed: 116.59 min
Epoch: 122/150 | Batch 0000/0092 | Cost: 2.8341
Time elapsed: 117.56 min
Epoch: 123/150 | Batch 0000/0092 | Cost: 2.6916
Time elapsed: 118.52 min
Epoch: 124/150 | Batch 0000/0092 | Cost: 2.6589
Time elapsed: 119.48 min
Epoch: 125/150 | Batch 0000/0092 | Cost: 2.6671
Time elapsed: 120.45 min
Epoch: 126/150 | Batch 0000/0092 | Cost: 2.5647
Time elapsed: 121.41 min
Epoch: 127/150 | Batch 0000/0092 | Cost: 2.5726
Time elapsed: 122.38 min
Epoch: 128/150 | Batch 0000/0092 | Cost: 2.5466
Time elapsed: 123.34 min
Epoch: 129/150 | Batch 0000/0092 | Cost: 2.4779
Time elapsed: 124.30 min
Epoch: 130/150 | Batch 0000/0092 | Cost: 2.5617
Time elapsed: 125.26 min
Epoch: 131/150 | Batch 0000/0092 | Cost: 2.4226
Time elapsed: 126.22 min
Epoch: 132/150 | Batch 0000/0092 | Cost: 2.3563
Time elapsed: 127.18 min
Epoch: 133/150 | Batch 0000/0092 | Cost: 2.4614
Time elapsed: 128.14 min
Epoch: 134/150 | Batch 0000/0092 | Cost: 2.3724
Time elapsed: 129.10 min
Epoch: 135/150 | Batch 0000/0092 | Cost: 2.5513
Time elapsed: 130.07 min
Epoch: 136/150 | Batch 0000/0092 | Cost: 3.1409
Time elapsed: 131.02 min
Epoch: 137/150 | Batch 0000/0092 | Cost: 3.1343
Time elapsed: 131.98 min
Epoch: 138/150 | Batch 0000/0092 | Cost: 3.0905
Time elapsed: 132.94 min
Epoch: 139/150 | Batch 0000/0092 | Cost: 2.8391
Time elapsed: 133.90 min
Epoch: 140/150 | Batch 0000/0092 | Cost: 2.6408
Time elapsed: 134.86 min
Epoch: 141/150 | Batch 0000/0092 | Cost: 2.4640
Time elapsed: 135.83 min
Epoch: 142/150 | Batch 0000/0092 | Cost: 2.4268
Time elapsed: 136.79 min
Epoch: 143/150 | Batch 0000/0092 | Cost: 2.4114
Time elapsed: 137.75 min
Epoch: 144/150 | Batch 0000/0092 | Cost: 2.3011
Time elapsed: 138.71 min
Epoch: 145/150 | Batch 0000/0092 | Cost: 2.2850
Time elapsed: 139.67 min
Epoch: 146/150 | Batch 0000/0092 | Cost: 2.3117
Time elapsed: 140.63 min
Epoch: 147/150 | Batch 0000/0092 | Cost: 2.3350
Time elapsed: 141.58 min
Epoch: 148/150 | Batch 0000/0092 | Cost: 2.1746
Time elapsed: 142.54 min
Epoch: 149/150 | Batch 0000/0092 | Cost: 2.3144
Time elapsed: 143.49 min
Epoch: 150/150 | Batch 0000/0092 | Cost: 2.2799
Time elapsed: 144.45 min

Evaluation

In [21]:
model.eval()
with torch.set_grad_enabled(False):  # save memory during inference

    train_mae, train_mse = compute_mae_and_mse(model, train_loader,
                                               device=DEVICE)
    test_mae, test_mse = compute_mae_and_mse(model, test_loader,
                                             device=DEVICE)

    s = 'MAE/RMSE: | Train: %.2f/%.2f | Test: %.2f/%.2f' % (
        train_mae, torch.sqrt(train_mse), test_mae, torch.sqrt(test_mse))
    print(s)

s = 'Total Training Time: %.2f min' % ((time.time() - start_time)/60)
print(s)
MAE/RMSE: | Train: 0.55/0.88 | Test: 3.38/4.71
Total Training Time: 145.23 min
In [22]:
%watermark -iv
numpy       1.15.4
pandas      0.23.4
torch       1.1.0
PIL.Image   5.3.0

In [ ]: