BentoML Example: PyTorch Fashion MNIST Classification¶
BentoML is an open source platform for machine learning model serving and deployment. In this project we will use BentoML to package the image classifier model, and build a containerized REST API model server.
This notebook demonstrates how to use BentoML to turn a PyTorch model into a docker image containing a REST API server serving this model, how to use your ML service built with BentoML as a CLI tool, and how to distribute it a pypi package.
This example was built based on https://github.com/baldassarreFe/zalando-pytorch/blob/master/notebooks/4.0-fb-autoencoder.ipynb, if you are familiar with this, jump start to Model Serving using BentoML
%reload_ext autoreload
%autoreload 2
%matplotlib inline
!pip install bentoml
!pip install torch torchvision sklearn pillow pandas numpy
import bentoml
import matplotlib.pyplot as plt
import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import transforms
from torch.autograd import Variable
from sklearn.manifold import TSNE
from sklearn.metrics import accuracy_score
Prepare Dataset¶
PyTorch supports FashionMNIST now, so we can import it directly.
from torchvision.datasets import FashionMNIST
FashionMNIST.classes
Load train and test set in batches of 1000.
The 28x28 images are scaled up to 29x29 so that combining convolutions and transposed convolutions would not chop off pixels from the reconstructed images.
batch_size = 1000
train_dataset = FashionMNIST(
'../data', train=True, download=True,
transform=transforms.Compose([transforms.CenterCrop((29, 29)), transforms.ToTensor()]))
train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=batch_size, shuffle=True)
test_dataset = FashionMNIST(
'../data', train=False, download=True,
transform=transforms.Compose([transforms.CenterCrop((29, 29)), transforms.ToTensor()]))
test_loader = torch.utils.data.DataLoader(test_dataset, batch_size=batch_size, shuffle=True)
Unsupervised reconstruction¶
Note that in this section we'll never use the image labels, the whole training is unsupervised.
Autoencoder¶
The two components of the autoencoder are defined subclassing nn.Module, that gives more flexibility than nn.Sequential.
Encoder¶
A series of convolutions with kernel_size=5 and stride=2 is used to squeeze the images into a volume of 40x1x1, then a fully connected layer turns this vector in a vector of size embedding_size, that can be specified externally.
Decoder¶
The decoder takes up from where the encoder left, first transforming back the embedding of size embedding_size into a volume of size 40x1x1, then applying a series of Transposed Convolutions to yield an image of the same size of the original input.
At this time we can show some images in this Dataloader.
class Encoder(nn.Module):
def __init__(self, embedding_size):
super(Encoder, self).__init__()
self.conv1 = nn.Conv2d(1, 10, kernel_size=5, stride=2)
self.conv2 = nn.Conv2d(10, 20, kernel_size=5, stride=2)
self.conv3 = nn.Conv2d(20, 40, kernel_size=5, stride=2)
self.fully = nn.Linear(40, embedding_size)
def forward(self, x):
# 1x29x29
x = torch.relu(self.conv1(x))
# 10x13x13
x = torch.relu(self.conv2(x))
# 20x5x5
x = torch.relu(self.conv3(x))
# 40x1x1
x = x.view(x.data.shape[0], 40)
# 40
x = self.fully(x)
# output_size
return x
class Decoder(nn.Module):
def __init__(self, input_size):
super(Decoder, self).__init__()
self.fully = nn.Linear(input_size, 40)
self.conv1 = nn.ConvTranspose2d(40, 20, kernel_size=5, stride=2)
self.conv2 = nn.ConvTranspose2d(20, 10, kernel_size=5, stride=2)
self.conv3 = nn.ConvTranspose2d(10, 1, kernel_size=5, stride=2)
def forward(self, x):
x = self.fully(x)
x = x.view(x.data.shape[0], 40, 1, 1)
x = torch.relu(self.conv1(x))
x = torch.relu(self.conv2(x))
x = torch.sigmoid(self.conv3(x))
return x
We are going to use an embedding size of 20, this number has no particular reason, except that it is in the same range of the number of classes. Naively, the network could learn to encode coarse-grained information (i.e. the kind of dress) in half of the embedding vector and then use the other half for fine-grained information.
embedding_size = 20
encoder = Encoder(embedding_size)
decoder = Decoder(embedding_size)
autoencoder = nn.Sequential(encoder, decoder)
Sanity check¶
A 29x29 black and white image passed through the autoencoder should give the same output dimension
x = Variable(torch.ones(1, 1, 29, 29))
e = encoder(x)
d = decoder(e)
print('Input\t ', list(x.data.shape))
print('Embedding', list(e.data.shape))
print('Output\t ', list(d.data.shape))
Training¶
autoencoder.train()
loss_fn = nn.MSELoss()
optimizer = optim.Adam(autoencoder.parameters())
epoch_loss = []
for epoch in range(15):
batch_loss = []
for batch_num, (data, _) in enumerate(train_loader):
data = Variable(data)
optimizer.zero_grad()
output = autoencoder(data)
loss = loss_fn(output, data)
loss.backward()
optimizer.step()
batch_loss.append(loss.item())
epoch_loss.append(sum(batch_loss) / len(batch_loss))
print('Epoch {}:\tloss {:.4f}'.format(epoch, epoch_loss[-1]))
plt.plot(epoch_loss)
plt.title('Final value {:.4f}'.format(epoch_loss[-1]))
plt.xlabel('Epoch')
plt.grid(True)
Evaluation¶
Reconsruction evaluation on a single batch
autoencoder.eval()
data, targets = next(test_loader.__iter__())
encodings = encoder(Variable(data))
outputs = decoder(encodings)
print('Test loss: {:.4f}'.format(loss_fn(outputs, Variable(data)).item()))
fig, axes = plt.subplots(8, 8, figsize=(16, 16))
axes = axes.ravel()
zip_these = axes[::2], axes[1::2], data.numpy().squeeze(), outputs.data.numpy().squeeze(), targets
for ax1, ax2, original, reconstructed, target in zip(*zip_these):
ax1.imshow(original, cmap='gray')
ax1.axis('off')
ax1.set_title(FashionMNIST.classes[target])
ax2.imshow(reconstructed, cmap='gray')
ax2.axis('off')
Embeddings¶
The embeddings are 20-dimensional, t-SNE is used to visualize them as clusters in 2D space.
Even though the autoencoder learned the embeddings in a completely unsupervised way we can observe the emergence of clusters:
- shoes (sandals, sneakers and ankle boot) are clustered together
- bags form a group on their own (they are the only images with a clear squared-rectangular shape)
- same goes for trousers, that form their own group
- all the others are quite mixed together, meaning that the network has learned the concept of clothes for the upper body, but is not able to tell a coat from a pullover
pca = TSNE(n_components=2)
encodings_2 = pca.fit_transform(encodings.data.numpy())
plt.figure(figsize=(10, 10))
for k in range(len(FashionMNIST.classes)):
class_indexes = (targets.numpy() == k)
plt.scatter(encodings_2[class_indexes, 0], encodings_2[class_indexes, 1], label=FashionMNIST.classes[k])
plt.legend();
Supervised classification¶
Once trained in an unsupervised fashion, the encoder module can be used to generate fashion embeddings (see what I did here?), that can then be used to train a simple classifier on the original labels.
Model¶
The weights of the encoder are freezed, so only the classifier will be trained.
(later on, when the classifier starts performing decently, we could unfreeze them and do some fine-tuning)
for param in encoder.parameters():
param.requires_grad = False
classifier = nn.Sequential(
encoder,
nn.Linear(embedding_size, 15),
nn.ReLU(),
nn.Linear(15, len(FashionMNIST.classes)),
nn.LogSoftmax()
)
Training¶
classifier.train()
loss_fn = nn.NLLLoss()
optimizer = optim.Adam([p for p in classifier.parameters() if p.requires_grad])
epoch_loss = []
for epoch in range(15):
batch_loss = []
for batch_num, (data, targets) in enumerate(train_loader):
data, targets = Variable(data), Variable(targets)
optimizer.zero_grad()
output = classifier(data)
loss = loss_fn(output, targets)
loss.backward()
optimizer.step()
batch_loss.append(loss.item())
epoch_loss.append(sum(batch_loss) / len(batch_loss))
accuracy = accuracy_score(targets.data.numpy(), output.data.numpy().argmax(axis=1))
print('Epoch {}:\tloss {:.4f}\taccuracy {:.2%}'.format(epoch, epoch_loss[-1], accuracy))
plt.plot(epoch_loss)
plt.title('Final value {:.4f}'.format(epoch_loss[-1]))
plt.xlabel('Epoch')
plt.grid(True)
Evaluation¶
Reconsruction evaluation on a single batch
classifier.eval()
data, targets = next(test_loader.__iter__())
outputs = classifier(Variable(data))
log_probs, output_classes = outputs.max(dim=1)
accuracy = accuracy_score(targets.numpy(), output_classes.data.numpy())
print('Accuracy: {:.2%}'.format(accuracy))
fig, axex = plt.subplots(8, 8, figsize=(16, 16))
zip_these = axex.ravel(), log_probs.data.exp(), output_classes.data, targets, data.numpy().squeeze()
for ax, prob, output_class, target, img in zip(*zip_these):
ax.imshow(img, cmap='gray' if output_class == target else 'autumn')
ax.axis('off')
ax.set_title('{} {:.1%}'.format(FashionMNIST.classes[output_class], prob))
Define BentoService for model serving¶
%%writefile pytorch_fashion_mnist.py
import bentoml
from PIL import Image
from torchvision import transforms
from bentoml.artifact import PytorchModelArtifact
from bentoml.handlers import ImageHandler
FASHION_MNIST_CLASSES = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
@bentoml.env(pip_dependencies=['torch', 'numpy', 'torchvision', 'scikit-learn'])
@bentoml.artifacts([PytorchModelArtifact('classifier')])
class PyTorchFashionClassifier(bentoml.BentoService):
@bentoml.api(ImageHandler, pilmode='L')
def predict(self, image):
img = Image.fromarray(image).resize((28, 28))
transform=transforms.Compose([transforms.CenterCrop((29, 29)), transforms.ToTensor()])
img_tensor = transform(img)
outputs = self.artifacts.classifier(img_tensor.unsqueeze(0))
_, output_classes = outputs.max(dim=1)
return FASHION_MNIST_CLASSES[output_classes]
Save BentoService¶
# 1) import the custom BentoService defined above
from pytorch_fashion_mnist import PyTorchFashionClassifier
# 2) `pack` it with required artifacts
bento_svc = PyTorchFashionClassifier()
bento_svc.pack('classifier', classifier)
# 3) save your BentoSerivce
saved_path = bento_svc.save()
Use bentoml get <BentoService_Name> to get list of service versions
!bentoml get PyTorchFashionClassifier
With version info, bentoml get display additional information and metadata
!bentoml get PyTorchFashionClassifier:20200213142355_A51D6E
Test and validate BentoService with bentoml run
!bentoml run PyTorchFashionClassifier:20200213142355_A51D6E predict --input sample_image.png
Model Serving via REST API¶
In your termnial, run the following command to start the REST API server:### Run REST API server
!bentoml serve {saved_path}
Call REST API from a client¶
Sending POST request from termnial:
curl -X POST "http://127.0.0.1:5000/predict" -F image=@sample_image.png
curl -X POST "http://127.0.0.1:5000/predict" -H "Content-Type: image/png" --data-binary @sample_image.png
Go visit http://127.0.0.1:5000/ from your browser, click /predict -> Try it out -> Choose File -> Execute to sumbit an image from your computer
Use BentoService as PyPI package¶
!pip install {saved_path}
!PyTorchFashionClassifier info
!PyTorchFashionClassifier run predict --input sample_image.png
Containerize REST API server with Docker¶
** Make sure you have docker installed, note that it is not available when running in Google Colaboratory
!cd {saved_path} && docker build . -t pytorch-fashion-mnist
!docker run -p 5000:5000 pytorch-fashion-mnist
Deploy BentoService as REST API server to the cloud¶
BentoML support deployment to multiply cloud provider services, such as AWS Lambda, AWS Sagemaker, Google Cloudrun and etc. You can find the full list and guide on the documentation site at https://docs.bentoml.org/en/latest/deployment/index.html
For this project, we are going to deploy to AWS Sagemaker
Deploy to Sagemaker with single command bentoml sagemaker deploy
!bentoml sagemaker deploy pytorch-fashion -b PyTorchFashionClassifier:20200213142355_A51D6E --api-name predict
bentoml sagemaker list lists all Sagemaker deployments
!bentoml sagemaker list
!bentoml sagemaker get pytorch-fashion
Test and validate Sagemaker deployment with aws sagemaker-runtime invoke-endpoint command
!aws sagemaker-runtime invoke-endpoint --endpoint-name bobo-pytorch-fashion \
--body fileb:///Users/bozhaoyu/src/bento_gallery/pytorch/fashion-mnist/sample_image.png \
--content-type image/png output.json && cat output.json
Clean up Sagemaker deployment with bentoml sagemaker delete
!bentoml sagemaker delete pytorch-fashion