BentoML Example: PyTorch Fashion MNIST Classification

BentoML is an open source platform for machine learning model serving and deployment. In this project we will use BentoML to package the image classifier model, and build a containerized REST API model server.

This notebook demonstrates how to use BentoML to turn a PyTorch model into a docker image containing a REST API server serving this model, how to use your ML service built with BentoML as a CLI tool, and how to distribute it a pypi package.

This example was built based on https://github.com/baldassarreFe/zalando-pytorch/blob/master/notebooks/4.0-fb-autoencoder.ipynb, if you are familiar with this, jump start to Model Serving using BentoML

Impression

In [1]:
%reload_ext autoreload
%autoreload 2
%matplotlib inline
In [ ]:
!pip install bentoml
!pip install torch torchvision sklearn pillow pandas numpy
In [2]:
import bentoml
import matplotlib.pyplot as plt

import torch
import torch.nn as nn
import torch.optim as optim

from torchvision import transforms
from torch.autograd import Variable

from sklearn.manifold import TSNE
from sklearn.metrics import accuracy_score

Prepare Dataset

PyTorch supports FashionMNIST now, so we can import it directly.

In [3]:
from torchvision.datasets import FashionMNIST
FashionMNIST.classes
Out[3]:
['T-shirt/top',
 'Trouser',
 'Pullover',
 'Dress',
 'Coat',
 'Sandal',
 'Shirt',
 'Sneaker',
 'Bag',
 'Ankle boot']

Load train and test set in batches of 1000.

The 28x28 images are scaled up to 29x29 so that combining convolutions and transposed convolutions would not chop off pixels from the reconstructed images.

In [4]:
batch_size = 1000

train_dataset = FashionMNIST(
    '../data', train=True, download=True, 
    transform=transforms.Compose([transforms.CenterCrop((29, 29)), transforms.ToTensor()]))
train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=batch_size, shuffle=True)

test_dataset = FashionMNIST(
    '../data', train=False, download=True, 
    transform=transforms.Compose([transforms.CenterCrop((29, 29)), transforms.ToTensor()]))
test_loader = torch.utils.data.DataLoader(test_dataset, batch_size=batch_size, shuffle=True)

Unsupervised reconstruction

Note that in this section we'll never use the image labels, the whole training is unsupervised.

Autoencoder

The two components of the autoencoder are defined subclassing nn.Module, that gives more flexibility than nn.Sequential.

Encoder

A series of convolutions with kernel_size=5 and stride=2 is used to squeeze the images into a volume of 40x1x1, then a fully connected layer turns this vector in a vector of size embedding_size, that can be specified externally.

Decoder

The decoder takes up from where the encoder left, first transforming back the embedding of size embedding_size into a volume of size 40x1x1, then applying a series of Transposed Convolutions to yield an image of the same size of the original input.

At this time we can show some images in this Dataloader.

In [5]:
class Encoder(nn.Module):
    def __init__(self, embedding_size):
        super(Encoder, self).__init__()
        self.conv1 = nn.Conv2d(1, 10, kernel_size=5, stride=2)
        self.conv2 = nn.Conv2d(10, 20, kernel_size=5, stride=2)
        self.conv3 = nn.Conv2d(20, 40, kernel_size=5, stride=2)
        self.fully = nn.Linear(40, embedding_size)

    def forward(self, x):
        # 1x29x29
        x = torch.relu(self.conv1(x))
        # 10x13x13
        x = torch.relu(self.conv2(x))
        # 20x5x5
        x = torch.relu(self.conv3(x))
        # 40x1x1
        x = x.view(x.data.shape[0], 40)
        # 40
        x = self.fully(x)
        # output_size
        return x

class Decoder(nn.Module):
    def __init__(self, input_size):
        super(Decoder, self).__init__()
        self.fully = nn.Linear(input_size, 40)
        self.conv1 = nn.ConvTranspose2d(40, 20, kernel_size=5, stride=2)
        self.conv2 = nn.ConvTranspose2d(20, 10, kernel_size=5, stride=2)
        self.conv3 = nn.ConvTranspose2d(10, 1, kernel_size=5, stride=2)
    
    def forward(self, x):
        x = self.fully(x)
        x = x.view(x.data.shape[0], 40, 1, 1)
        x = torch.relu(self.conv1(x))
        x = torch.relu(self.conv2(x))
        x = torch.sigmoid(self.conv3(x))
        return x

We are going to use an embedding size of 20, this number has no particular reason, except that it is in the same range of the number of classes. Naively, the network could learn to encode coarse-grained information (i.e. the kind of dress) in half of the embedding vector and then use the other half for fine-grained information.

In [6]:
embedding_size = 20
encoder = Encoder(embedding_size)
decoder = Decoder(embedding_size)

autoencoder = nn.Sequential(encoder, decoder)

Sanity check

A 29x29 black and white image passed through the autoencoder should give the same output dimension

In [7]:
x = Variable(torch.ones(1, 1, 29, 29))
e = encoder(x)
d = decoder(e)

print('Input\t ', list(x.data.shape))
print('Embedding', list(e.data.shape))
print('Output\t ', list(d.data.shape))
Input	  [1, 1, 29, 29]
Embedding [1, 20]
Output	  [1, 1, 29, 29]

Training

In [8]:
autoencoder.train()

loss_fn = nn.MSELoss()
optimizer = optim.Adam(autoencoder.parameters())
epoch_loss = []

for epoch in range(15):
    batch_loss = []
    for batch_num, (data, _) in enumerate(train_loader):
        data = Variable(data)
        optimizer.zero_grad()
        output = autoencoder(data)
        loss = loss_fn(output, data)
        loss.backward()
        optimizer.step()
        batch_loss.append(loss.item())
    epoch_loss.append(sum(batch_loss) / len(batch_loss))
    print('Epoch {}:\tloss {:.4f}'.format(epoch, epoch_loss[-1]))
Epoch 0:	loss 0.1282
Epoch 1:	loss 0.0558
Epoch 2:	loss 0.0362
Epoch 3:	loss 0.0286
Epoch 4:	loss 0.0258
Epoch 5:	loss 0.0245
Epoch 6:	loss 0.0237
Epoch 7:	loss 0.0229
Epoch 8:	loss 0.0223
Epoch 9:	loss 0.0217
Epoch 10:	loss 0.0212
Epoch 11:	loss 0.0207
Epoch 12:	loss 0.0202
Epoch 13:	loss 0.0198
Epoch 14:	loss 0.0194
In [9]:
plt.plot(epoch_loss)
plt.title('Final value {:.4f}'.format(epoch_loss[-1]))
plt.xlabel('Epoch')
plt.grid(True)

Evaluation

Reconsruction evaluation on a single batch

In [10]:
autoencoder.eval()
data, targets = next(test_loader.__iter__())
encodings = encoder(Variable(data))
outputs = decoder(encodings)

print('Test loss: {:.4f}'.format(loss_fn(outputs, Variable(data)).item()))
Test loss: 0.0204
In [11]:
fig, axes = plt.subplots(8, 8, figsize=(16, 16))
axes = axes.ravel()

zip_these = axes[::2], axes[1::2], data.numpy().squeeze(), outputs.data.numpy().squeeze(), targets
for ax1, ax2, original, reconstructed, target in zip(*zip_these):
    ax1.imshow(original, cmap='gray')
    ax1.axis('off')
    ax1.set_title(FashionMNIST.classes[target])
    ax2.imshow(reconstructed, cmap='gray')
    ax2.axis('off')

Embeddings

The embeddings are 20-dimensional, t-SNE is used to visualize them as clusters in 2D space.

Even though the autoencoder learned the embeddings in a completely unsupervised way we can observe the emergence of clusters:

  • shoes (sandals, sneakers and ankle boot) are clustered together
  • bags form a group on their own (they are the only images with a clear squared-rectangular shape)
  • same goes for trousers, that form their own group
  • all the others are quite mixed together, meaning that the network has learned the concept of clothes for the upper body, but is not able to tell a coat from a pullover
In [12]:
pca = TSNE(n_components=2)
encodings_2 = pca.fit_transform(encodings.data.numpy())
plt.figure(figsize=(10, 10))
for k in range(len(FashionMNIST.classes)):
    class_indexes = (targets.numpy() == k)
    plt.scatter(encodings_2[class_indexes, 0], encodings_2[class_indexes, 1], label=FashionMNIST.classes[k])
plt.legend();

Supervised classification

Once trained in an unsupervised fashion, the encoder module can be used to generate fashion embeddings (see what I did here?), that can then be used to train a simple classifier on the original labels.

Model

The weights of the encoder are freezed, so only the classifier will be trained.

(later on, when the classifier starts performing decently, we could unfreeze them and do some fine-tuning)

In [13]:
for param in encoder.parameters():
    param.requires_grad = False

classifier = nn.Sequential(
    encoder, 
    nn.Linear(embedding_size, 15),
    nn.ReLU(),
    nn.Linear(15, len(FashionMNIST.classes)),
    nn.LogSoftmax()
)

Training

In [14]:
classifier.train()

loss_fn = nn.NLLLoss()
optimizer = optim.Adam([p for p in classifier.parameters() if p.requires_grad])
epoch_loss = []

for epoch in range(15):
    batch_loss = []
    for batch_num, (data, targets) in enumerate(train_loader):
        data, targets = Variable(data), Variable(targets)
        optimizer.zero_grad()
        output = classifier(data)
        loss = loss_fn(output, targets)
        loss.backward()
        optimizer.step()
        batch_loss.append(loss.item())
    epoch_loss.append(sum(batch_loss) / len(batch_loss))
    accuracy = accuracy_score(targets.data.numpy(), output.data.numpy().argmax(axis=1))
    print('Epoch {}:\tloss {:.4f}\taccuracy {:.2%}'.format(epoch, epoch_loss[-1], accuracy))
/usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages/torch/nn/modules/container.py:92: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.
  input = module(input)
Epoch 0:	loss 4.1516	accuracy 37.70%
Epoch 1:	loss 1.2190	accuracy 60.90%
Epoch 2:	loss 0.9746	accuracy 65.60%
Epoch 3:	loss 0.8748	accuracy 68.40%
Epoch 4:	loss 0.8189	accuracy 70.70%
Epoch 5:	loss 0.7854	accuracy 72.90%
Epoch 6:	loss 0.7626	accuracy 73.80%
Epoch 7:	loss 0.7472	accuracy 72.50%
Epoch 8:	loss 0.7336	accuracy 73.30%
Epoch 9:	loss 0.7252	accuracy 74.20%
Epoch 10:	loss 0.7169	accuracy 76.10%
Epoch 11:	loss 0.7125	accuracy 74.90%
Epoch 12:	loss 0.7065	accuracy 74.20%
Epoch 13:	loss 0.7030	accuracy 71.70%
Epoch 14:	loss 0.7003	accuracy 76.70%
In [15]:
plt.plot(epoch_loss)
plt.title('Final value {:.4f}'.format(epoch_loss[-1]))
plt.xlabel('Epoch')
plt.grid(True)

Evaluation

Reconsruction evaluation on a single batch

In [16]:
classifier.eval()
data, targets = next(test_loader.__iter__())
outputs = classifier(Variable(data))
log_probs, output_classes = outputs.max(dim=1)

accuracy = accuracy_score(targets.numpy(), output_classes.data.numpy())
print('Accuracy: {:.2%}'.format(accuracy))
Accuracy: 74.60%
In [17]:
fig, axex = plt.subplots(8, 8, figsize=(16, 16))

zip_these = axex.ravel(), log_probs.data.exp(), output_classes.data, targets, data.numpy().squeeze()
for ax, prob, output_class, target, img in zip(*zip_these):
    ax.imshow(img, cmap='gray' if output_class == target else 'autumn')
    ax.axis('off')
    ax.set_title('{} {:.1%}'.format(FashionMNIST.classes[output_class], prob))

Define BentoService for model serving

In [21]:
%%writefile pytorch_fashion_mnist.py

import bentoml
from PIL import Image
from torchvision import transforms

from bentoml.artifact import PytorchModelArtifact
from bentoml.handlers import ImageHandler


FASHION_MNIST_CLASSES = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']


@bentoml.env(pip_dependencies=['torch', 'numpy', 'torchvision', 'scikit-learn'])
@bentoml.artifacts([PytorchModelArtifact('classifier')])
class PyTorchFashionClassifier(bentoml.BentoService):

    @bentoml.api(ImageHandler, pilmode='L')
    def predict(self, image):
        img = Image.fromarray(image).resize((28, 28))
        transform=transforms.Compose([transforms.CenterCrop((29, 29)), transforms.ToTensor()])
        img_tensor = transform(img)
        outputs = self.artifacts.classifier(img_tensor.unsqueeze(0))        
        _, output_classes = outputs.max(dim=1)
        
        return FASHION_MNIST_CLASSES[output_classes]
Overwriting pytorch_fashion_mnist.py

Save BentoService

In [22]:
# 1) import the custom BentoService defined above
from pytorch_fashion_mnist import PyTorchFashionClassifier

# 2) `pack` it with required artifacts
bento_svc = PyTorchFashionClassifier()
bento_svc.pack('classifier', classifier)

# 3) save your BentoSerivce
saved_path = bento_svc.save()
[2020-02-13 14:23:55,951] WARNING - BentoML local changes detected - Local BentoML repository including all code changes will be bundled together with the BentoService bundle. When used with docker, the base docker image will be default to same version as last PyPI release at version: 0.6.2. You can also force bentoml to use a specific version for deploying your BentoService bundle, by setting the config 'core/bentoml_deploy_version' to a pinned version or your custom BentoML on github, e.g.:'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
[2020-02-13 14:23:55,977] WARNING - BentoML local changes detected - Local BentoML repository including all code changes will be bundled together with the BentoService bundle. When used with docker, the base docker image will be default to same version as last PyPI release at version: 0.6.2. You can also force bentoml to use a specific version for deploying your BentoService bundle, by setting the config 'core/bentoml_deploy_version' to a pinned version or your custom BentoML on github, e.g.:'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
[2020-02-13 14:24:21,764] WARNING - BentoML local changes detected - Local BentoML repository including all code changes will be bundled together with the BentoService bundle. When used with docker, the base docker image will be default to same version as last PyPI release at version: 0.6.2. You can also force bentoml to use a specific version for deploying your BentoService bundle, by setting the config 'core/bentoml_deploy_version' to a pinned version or your custom BentoML on github, e.g.:'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
running sdist
running egg_info
writing BentoML.egg-info/PKG-INFO
writing dependency_links to BentoML.egg-info/dependency_links.txt
writing entry points to BentoML.egg-info/entry_points.txt
writing requirements to BentoML.egg-info/requires.txt
writing top-level names to BentoML.egg-info/top_level.txt
reading manifest file 'BentoML.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching 'examples'
no previously-included directories found matching 'tests'
no previously-included directories found matching 'docs'
no previously-included directories found matching 'scripts'
warning: no previously-included files matching '*~' found anywhere in distribution
warning: no previously-included files matching '*.pyo' found anywhere in distribution
warning: no previously-included files matching '.git' found anywhere in distribution
warning: no previously-included files matching '.ipynb_checkpoints' found anywhere in distribution
warning: no previously-included files matching '__pycache__' found anywhere in distribution
writing manifest file 'BentoML.egg-info/SOURCES.txt'
running check
warning: check: missing meta-data: if 'author' supplied, 'author_email' must be supplied too

creating BentoML-0.6.2+8.gd95a887
creating BentoML-0.6.2+8.gd95a887/BentoML.egg-info
creating BentoML-0.6.2+8.gd95a887/bentoml
creating BentoML-0.6.2+8.gd95a887/bentoml/artifact
creating BentoML-0.6.2+8.gd95a887/bentoml/bundler
creating BentoML-0.6.2+8.gd95a887/bentoml/cli
creating BentoML-0.6.2+8.gd95a887/bentoml/clipper
creating BentoML-0.6.2+8.gd95a887/bentoml/configuration
creating BentoML-0.6.2+8.gd95a887/bentoml/deployment
creating BentoML-0.6.2+8.gd95a887/bentoml/deployment/aws_lambda
creating BentoML-0.6.2+8.gd95a887/bentoml/deployment/sagemaker
creating BentoML-0.6.2+8.gd95a887/bentoml/handlers
creating BentoML-0.6.2+8.gd95a887/bentoml/marshal
creating BentoML-0.6.2+8.gd95a887/bentoml/migrations
creating BentoML-0.6.2+8.gd95a887/bentoml/migrations/versions
creating BentoML-0.6.2+8.gd95a887/bentoml/proto
creating BentoML-0.6.2+8.gd95a887/bentoml/repository
creating BentoML-0.6.2+8.gd95a887/bentoml/server
creating BentoML-0.6.2+8.gd95a887/bentoml/server/static
creating BentoML-0.6.2+8.gd95a887/bentoml/utils
creating BentoML-0.6.2+8.gd95a887/bentoml/utils/validator
creating BentoML-0.6.2+8.gd95a887/bentoml/yatai
creating BentoML-0.6.2+8.gd95a887/bentoml/yatai/client
copying files to BentoML-0.6.2+8.gd95a887...
copying LICENSE -> BentoML-0.6.2+8.gd95a887
copying MANIFEST.in -> BentoML-0.6.2+8.gd95a887
copying README.md -> BentoML-0.6.2+8.gd95a887
copying pyproject.toml -> BentoML-0.6.2+8.gd95a887
copying setup.cfg -> BentoML-0.6.2+8.gd95a887
copying setup.py -> BentoML-0.6.2+8.gd95a887
copying versioneer.py -> BentoML-0.6.2+8.gd95a887
copying BentoML.egg-info/PKG-INFO -> BentoML-0.6.2+8.gd95a887/BentoML.egg-info
copying BentoML.egg-info/SOURCES.txt -> BentoML-0.6.2+8.gd95a887/BentoML.egg-info
copying BentoML.egg-info/dependency_links.txt -> BentoML-0.6.2+8.gd95a887/BentoML.egg-info
copying BentoML.egg-info/entry_points.txt -> BentoML-0.6.2+8.gd95a887/BentoML.egg-info
copying BentoML.egg-info/requires.txt -> BentoML-0.6.2+8.gd95a887/BentoML.egg-info
copying BentoML.egg-info/top_level.txt -> BentoML-0.6.2+8.gd95a887/BentoML.egg-info
copying bentoml/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml
copying bentoml/_version.py -> BentoML-0.6.2+8.gd95a887/bentoml
copying bentoml/alembic.ini -> BentoML-0.6.2+8.gd95a887/bentoml
copying bentoml/db.py -> BentoML-0.6.2+8.gd95a887/bentoml
copying bentoml/exceptions.py -> BentoML-0.6.2+8.gd95a887/bentoml
copying bentoml/service.py -> BentoML-0.6.2+8.gd95a887/bentoml
copying bentoml/service_env.py -> BentoML-0.6.2+8.gd95a887/bentoml
copying bentoml/artifact/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/fastai_model_artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/h2o_model_artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/keras_model_artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/lightgbm_model_artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/pickle_artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/pytorch_model_artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/sklearn_model_artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/text_file_artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/tf_savedmodel_artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/xgboost_model_artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/bundler/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/bundler
copying bentoml/bundler/bundler.py -> BentoML-0.6.2+8.gd95a887/bentoml/bundler
copying bentoml/bundler/config.py -> BentoML-0.6.2+8.gd95a887/bentoml/bundler
copying bentoml/bundler/loader.py -> BentoML-0.6.2+8.gd95a887/bentoml/bundler
copying bentoml/bundler/py_module_utils.py -> BentoML-0.6.2+8.gd95a887/bentoml/bundler
copying bentoml/bundler/templates.py -> BentoML-0.6.2+8.gd95a887/bentoml/bundler
copying bentoml/bundler/utils.py -> BentoML-0.6.2+8.gd95a887/bentoml/bundler
copying bentoml/cli/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/cli
copying bentoml/cli/aws_lambda.py -> BentoML-0.6.2+8.gd95a887/bentoml/cli
copying bentoml/cli/aws_sagemaker.py -> BentoML-0.6.2+8.gd95a887/bentoml/cli
copying bentoml/cli/bento.py -> BentoML-0.6.2+8.gd95a887/bentoml/cli
copying bentoml/cli/click_utils.py -> BentoML-0.6.2+8.gd95a887/bentoml/cli
copying bentoml/cli/config.py -> BentoML-0.6.2+8.gd95a887/bentoml/cli
copying bentoml/cli/deployment.py -> BentoML-0.6.2+8.gd95a887/bentoml/cli
copying bentoml/cli/utils.py -> BentoML-0.6.2+8.gd95a887/bentoml/cli
copying bentoml/clipper/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/clipper
copying bentoml/configuration/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/configuration
copying bentoml/configuration/configparser.py -> BentoML-0.6.2+8.gd95a887/bentoml/configuration
copying bentoml/configuration/default_bentoml.cfg -> BentoML-0.6.2+8.gd95a887/bentoml/configuration
copying bentoml/deployment/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment
copying bentoml/deployment/operator.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment
copying bentoml/deployment/store.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment
copying bentoml/deployment/utils.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment
copying bentoml/deployment/aws_lambda/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment/aws_lambda
copying bentoml/deployment/aws_lambda/download_extra_resources.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment/aws_lambda
copying bentoml/deployment/aws_lambda/lambda_app.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment/aws_lambda
copying bentoml/deployment/aws_lambda/utils.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment/aws_lambda
copying bentoml/deployment/sagemaker/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment/sagemaker
copying bentoml/deployment/sagemaker/sagemaker_nginx.conf -> BentoML-0.6.2+8.gd95a887/bentoml/deployment/sagemaker
copying bentoml/deployment/sagemaker/sagemaker_serve.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment/sagemaker
copying bentoml/deployment/sagemaker/sagemaker_wsgi.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment/sagemaker
copying bentoml/handlers/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/handlers
copying bentoml/handlers/base_handlers.py -> BentoML-0.6.2+8.gd95a887/bentoml/handlers
copying bentoml/handlers/clipper_handler.py -> BentoML-0.6.2+8.gd95a887/bentoml/handlers
copying bentoml/handlers/dataframe_handler.py -> BentoML-0.6.2+8.gd95a887/bentoml/handlers
copying bentoml/handlers/fastai_image_handler.py -> BentoML-0.6.2+8.gd95a887/bentoml/handlers
copying bentoml/handlers/image_handler.py -> BentoML-0.6.2+8.gd95a887/bentoml/handlers
copying bentoml/handlers/json_handler.py -> BentoML-0.6.2+8.gd95a887/bentoml/handlers
copying bentoml/handlers/pytorch_tensor_handler.py -> BentoML-0.6.2+8.gd95a887/bentoml/handlers
copying bentoml/handlers/tensorflow_tensor_handler.py -> BentoML-0.6.2+8.gd95a887/bentoml/handlers
copying bentoml/handlers/utils.py -> BentoML-0.6.2+8.gd95a887/bentoml/handlers
copying bentoml/marshal/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/marshal
copying bentoml/marshal/marshal.py -> BentoML-0.6.2+8.gd95a887/bentoml/marshal
copying bentoml/marshal/utils.py -> BentoML-0.6.2+8.gd95a887/bentoml/marshal
copying bentoml/migrations/README -> BentoML-0.6.2+8.gd95a887/bentoml/migrations
copying bentoml/migrations/env.py -> BentoML-0.6.2+8.gd95a887/bentoml/migrations
copying bentoml/migrations/script.py.mako -> BentoML-0.6.2+8.gd95a887/bentoml/migrations
copying bentoml/migrations/versions/a6b00ae45279_add_last_updated_at_for_deployments.py -> BentoML-0.6.2+8.gd95a887/bentoml/migrations/versions
copying bentoml/proto/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/proto
copying bentoml/proto/deployment_pb2.py -> BentoML-0.6.2+8.gd95a887/bentoml/proto
copying bentoml/proto/repository_pb2.py -> BentoML-0.6.2+8.gd95a887/bentoml/proto
copying bentoml/proto/status_pb2.py -> BentoML-0.6.2+8.gd95a887/bentoml/proto
copying bentoml/proto/yatai_service_pb2.py -> BentoML-0.6.2+8.gd95a887/bentoml/proto
copying bentoml/proto/yatai_service_pb2_grpc.py -> BentoML-0.6.2+8.gd95a887/bentoml/proto
copying bentoml/repository/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/repository
copying bentoml/repository/metadata_store.py -> BentoML-0.6.2+8.gd95a887/bentoml/repository
copying bentoml/server/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/server
copying bentoml/server/bento_api_server.py -> BentoML-0.6.2+8.gd95a887/bentoml/server
copying bentoml/server/bento_sagemaker_server.py -> BentoML-0.6.2+8.gd95a887/bentoml/server
copying bentoml/server/gunicorn_config.py -> BentoML-0.6.2+8.gd95a887/bentoml/server
copying bentoml/server/gunicorn_server.py -> BentoML-0.6.2+8.gd95a887/bentoml/server
copying bentoml/server/marshal_server.py -> BentoML-0.6.2+8.gd95a887/bentoml/server
copying bentoml/server/middlewares.py -> BentoML-0.6.2+8.gd95a887/bentoml/server
copying bentoml/server/utils.py -> BentoML-0.6.2+8.gd95a887/bentoml/server
copying bentoml/server/static/swagger-ui-bundle.js -> BentoML-0.6.2+8.gd95a887/bentoml/server/static
copying bentoml/server/static/swagger-ui.css -> BentoML-0.6.2+8.gd95a887/bentoml/server/static
copying bentoml/utils/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/utils
copying bentoml/utils/cloudpickle.py -> BentoML-0.6.2+8.gd95a887/bentoml/utils
copying bentoml/utils/hybirdmethod.py -> BentoML-0.6.2+8.gd95a887/bentoml/utils
copying bentoml/utils/log.py -> BentoML-0.6.2+8.gd95a887/bentoml/utils
copying bentoml/utils/s3.py -> BentoML-0.6.2+8.gd95a887/bentoml/utils
copying bentoml/utils/tempdir.py -> BentoML-0.6.2+8.gd95a887/bentoml/utils
copying bentoml/utils/usage_stats.py -> BentoML-0.6.2+8.gd95a887/bentoml/utils
copying bentoml/utils/validator/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/utils/validator
copying bentoml/yatai/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/yatai
copying bentoml/yatai/deployment_utils.py -> BentoML-0.6.2+8.gd95a887/bentoml/yatai
copying bentoml/yatai/status.py -> BentoML-0.6.2+8.gd95a887/bentoml/yatai
copying bentoml/yatai/yatai_service_impl.py -> BentoML-0.6.2+8.gd95a887/bentoml/yatai
copying bentoml/yatai/client/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/yatai/client
copying bentoml/yatai/client/bento_repository_api.py -> BentoML-0.6.2+8.gd95a887/bentoml/yatai/client
copying bentoml/yatai/client/deployment_api.py -> BentoML-0.6.2+8.gd95a887/bentoml/yatai/client
Writing BentoML-0.6.2+8.gd95a887/setup.cfg
UPDATING BentoML-0.6.2+8.gd95a887/bentoml/_version.py
set BentoML-0.6.2+8.gd95a887/bentoml/_version.py to '0.6.2+8.gd95a887'
Creating tar archive
removing 'BentoML-0.6.2+8.gd95a887' (and everything under it)
[2020-02-13 14:24:23,185] INFO - BentoService bundle 'PyTorchFashionClassifier:20200213142355_A51D6E' created at: /private/var/folders/kn/xnc9k74x03567n1mx2tfqnpr0000gn/T/bentoml-temp-1ex_gaj_
[2020-02-13 14:24:23,187] WARNING - BentoML local changes detected - Local BentoML repository including all code changes will be bundled together with the BentoService bundle. When used with docker, the base docker image will be default to same version as last PyPI release at version: 0.6.2. You can also force bentoml to use a specific version for deploying your BentoService bundle, by setting the config 'core/bentoml_deploy_version' to a pinned version or your custom BentoML on github, e.g.:'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
[2020-02-13 14:24:23,199] WARNING - Saved BentoService bundle version mismatch: loading BentoServie bundle create with BentoML version 0.6.2,  but loading from BentoML version 0.6.2+8.gd95a887
[2020-02-13 14:24:23,254] INFO - BentoService bundle 'PyTorchFashionClassifier:20200213142355_A51D6E' created at: /Users/bozhaoyu/bentoml/repository/PyTorchFashionClassifier/20200213142355_A51D6E

Use bentoml get <BentoService_Name> to get list of service versions

In [23]:
!bentoml get PyTorchFashionClassifier
BENTO_SERVICE                                   AGE            APIS                   ARTIFACTS
PyTorchFashionClassifier:20200213142355_A51D6E  47.93 seconds  predict<ImageHandler>  classifier<PytorchModelArtifact>

With version info, bentoml get display additional information and metadata

In [24]:
!bentoml get PyTorchFashionClassifier:20200213142355_A51D6E
{
  "name": "PyTorchFashionClassifier",
  "version": "20200213142355_A51D6E",
  "uri": {
    "type": "LOCAL",
    "uri": "/Users/bozhaoyu/bentoml/repository/PyTorchFashionClassifier/20200213142355_A51D6E"
  },
  "bentoServiceMetadata": {
    "name": "PyTorchFashionClassifier",
    "version": "20200213142355_A51D6E",
    "createdAt": "2020-02-13T22:24:21.765194Z",
    "env": {
      "condaEnv": "name: bentoml-PyTorchFashionClassifier\nchannels:\n- defaults\ndependencies:\n- python=3.7.3\n- pip\n",
      "pipDependencies": "bentoml==0.6.2\ntorch\nnumpy\ntorchvision\nscikit-learn\nimageio",
      "pythonVersion": "3.7.3"
    },
    "artifacts": [
      {
        "name": "classifier",
        "artifactType": "PytorchModelArtifact"
      }
    ],
    "apis": [
      {
        "name": "predict",
        "handlerType": "ImageHandler",
        "docs": "BentoService API",
        "handlerConfig": {
          "accept_image_formats": [
            ".jpg",
            ".png",
            ".jpeg",
            ".tiff",
            ".webp",
            ".bmp"
          ],
          "pilmode": "L",
          "input_names": [
            "image"
          ]
        }
      }
    ]
  }
}

Test and validate BentoService with bentoml run

In [25]:
!bentoml run PyTorchFashionClassifier:20200213142355_A51D6E predict --input sample_image.png
[2020-02-13 14:26:27,300] WARNING - BentoML local changes detected - Local BentoML repository including all code changes will be bundled together with the BentoService bundle. When used with docker, the base docker image will be default to same version as last PyPI release at version: 0.6.2. You can also force bentoml to use a specific version for deploying your BentoService bundle, by setting the config 'core/bentoml_deploy_version' to a pinned version or your custom BentoML on github, e.g.:'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
[2020-02-13 14:26:27,314] WARNING - Saved BentoService bundle version mismatch: loading BentoServie bundle create with BentoML version 0.6.2,  but loading from BentoML version 0.6.2+8.gd95a887
[2020-02-13 14:26:27,867] WARNING - BentoML local changes detected - Local BentoML repository including all code changes will be bundled together with the BentoService bundle. When used with docker, the base docker image will be default to same version as last PyPI release at version: 0.6.2. You can also force bentoml to use a specific version for deploying your BentoService bundle, by setting the config 'core/bentoml_deploy_version' to a pinned version or your custom BentoML on github, e.g.:'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
[2020-02-13 14:26:27,871] WARNING - BentoML local changes detected - Local BentoML repository including all code changes will be bundled together with the BentoService bundle. When used with docker, the base docker image will be default to same version as last PyPI release at version: 0.6.2. You can also force bentoml to use a specific version for deploying your BentoService bundle, by setting the config 'core/bentoml_deploy_version' to a pinned version or your custom BentoML on github, e.g.:'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
/usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages/torch/nn/modules/container.py:92: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.
  input = module(input)
Bag

Model Serving via REST API

In your termnial, run the following command to start the REST API server:### Run REST API server

In [20]:
!bentoml serve {saved_path}
 * Serving Flask app "FashionMNISTServicer" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
127.0.0.1 - - [19/Sep/2019 12:52:50] "GET / HTTP/1.1" 200 -
/Users/chaoyuyang/anaconda3/envs/bentoml-dev/lib/python3.7/site-packages/torch/nn/modules/container.py:92: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.
  input = module(input)
127.0.0.1 - - [19/Sep/2019 12:52:58] "POST /predict HTTP/1.1" 200 -
127.0.0.1 - - [19/Sep/2019 12:53:01] "GET /docs.json HTTP/1.1" 200 -
127.0.0.1 - - [19/Sep/2019 12:53:04] "POST /predict HTTP/1.1" 200 -
^C

Call REST API from a client

Sending POST request from termnial:

curl -X POST "http://127.0.0.1:5000/predict" -F image=@sample_image.png
curl -X POST "http://127.0.0.1:5000/predict" -H "Content-Type: image/png" --data-binary @sample_image.png

Go visit http://127.0.0.1:5000/ from your browser, click /predict -> Try it out -> Choose File -> Execute to sumbit an image from your computer

image.png

Use BentoService as PyPI package

In [26]:
!pip install {saved_path}
Processing /Users/bozhaoyu/bentoml/repository/PyTorchFashionClassifier/20200213142355_A51D6E
Requirement already satisfied: bentoml==0.6.2 in /Users/bozhaoyu/src/bento (from PyTorchFashionClassifier===20200213142355-A51D6E) (0.6.2+8.gd95a887)
Requirement already satisfied: torch in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from PyTorchFashionClassifier===20200213142355-A51D6E) (1.0.1.post2)
Requirement already satisfied: numpy in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from PyTorchFashionClassifier===20200213142355-A51D6E) (1.16.4)
Requirement already satisfied: torchvision in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from PyTorchFashionClassifier===20200213142355-A51D6E) (0.2.2)
Requirement already satisfied: scikit-learn in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from PyTorchFashionClassifier===20200213142355-A51D6E) (0.22)
Requirement already satisfied: imageio in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from PyTorchFashionClassifier===20200213142355-A51D6E) (2.5.0)
Requirement already satisfied: ruamel.yaml>=0.15.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (0.16.5)
Requirement already satisfied: flask in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (1.0.4)
Requirement already satisfied: gunicorn in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (19.9.0)
Requirement already satisfied: click>=7.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (7.0)
Requirement already satisfied: pandas in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (0.24.2)
Requirement already satisfied: prometheus_client in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (0.7.1)
Requirement already satisfied: python-json-logger in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (0.1.11)
Requirement already satisfied: boto3 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (1.10.42)
Requirement already satisfied: requests in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (2.22.0)
Requirement already satisfied: packaging in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (19.0)
Requirement already satisfied: docker in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (4.1.0)
Requirement already satisfied: configparser in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (3.7.4)
Requirement already satisfied: sqlalchemy>=1.3.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (1.3.6)
Requirement already satisfied: protobuf>=3.6.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (3.7.0)
Requirement already satisfied: grpcio in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (1.22.0)
Requirement already satisfied: cerberus in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (1.3.1)
Requirement already satisfied: tabulate in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (0.8.3)
Requirement already satisfied: humanfriendly in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (4.18)
Requirement already satisfied: alembic in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (1.2.1)
Requirement already satisfied: aiohttp in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (3.6.2)
Requirement already satisfied: python-dateutil<2.8.1,>=2.1 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (2.8.0)
Requirement already satisfied: six in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from torchvision->PyTorchFashionClassifier===20200213142355-A51D6E) (1.14.0)
Requirement already satisfied: pillow>=4.1.1 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from torchvision->PyTorchFashionClassifier===20200213142355-A51D6E) (6.0.0)
Requirement already satisfied: scipy>=0.17.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from scikit-learn->PyTorchFashionClassifier===20200213142355-A51D6E) (1.2.1)
Requirement already satisfied: joblib>=0.11 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from scikit-learn->PyTorchFashionClassifier===20200213142355-A51D6E) (0.14.1)
Requirement already satisfied: ruamel.yaml.clib>=0.1.2; platform_python_implementation == "CPython" and python_version < "3.8" in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from ruamel.yaml>=0.15.0->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (0.2.0)
Requirement already satisfied: Werkzeug>=0.14 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from flask->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (0.15.4)
Requirement already satisfied: Jinja2>=2.10 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from flask->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (2.10.1)
Requirement already satisfied: itsdangerous>=0.24 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from flask->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (1.1.0)
Requirement already satisfied: pytz>=2011k in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from pandas->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (2019.1)
Requirement already satisfied: s3transfer<0.3.0,>=0.2.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from boto3->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (0.2.1)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from boto3->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (0.9.4)
Requirement already satisfied: botocore<1.14.0,>=1.13.42 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from boto3->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (1.13.42)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from requests->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (1.24.1)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from requests->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (2.8)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from requests->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from requests->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (2019.11.28)
Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from packaging->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (2.4.0)
Requirement already satisfied: websocket-client>=0.32.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from docker->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (0.57.0)
Requirement already satisfied: setuptools in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from protobuf>=3.6.0->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (45.1.0.post20200119)
Requirement already satisfied: Mako in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from alembic->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (1.1.0)
Requirement already satisfied: python-editor>=0.3 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from alembic->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (1.0.4)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from aiohttp->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (19.1.0)
Requirement already satisfied: async-timeout<4.0,>=3.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from aiohttp->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (3.0.1)
Requirement already satisfied: multidict<5.0,>=4.5 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from aiohttp->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (4.7.4)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from aiohttp->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (1.4.2)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from Jinja2>=2.10->flask->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (1.1.1)
Requirement already satisfied: docutils<0.16,>=0.10 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from botocore<1.14.0,>=1.13.42->boto3->bentoml==0.6.2->PyTorchFashionClassifier===20200213142355-A51D6E) (0.15.2)
Building wheels for collected packages: PyTorchFashionClassifier
  Building wheel for PyTorchFashionClassifier (setup.py) ... done
  Created wheel for PyTorchFashionClassifier: filename=PyTorchFashionClassifier-20200213142355_A51D6E-py3-none-any.whl size=227357 sha256=f6392e9808e635494a70bc99f38324bd6a95e87faf78e09a98bf9dabd1f83358
  Stored in directory: /private/var/folders/kn/xnc9k74x03567n1mx2tfqnpr0000gn/T/pip-ephem-wheel-cache-g_t2q7hh/wheels/8d/74/85/123929307607d586ca80844b9f8a112a628d7f7e1a273c94dc
Successfully built PyTorchFashionClassifier
Installing collected packages: PyTorchFashionClassifier
Successfully installed PyTorchFashionClassifier-20200213142355-A51D6E
In [27]:
!PyTorchFashionClassifier info
[2020-02-13 14:28:34,473] WARNING - BentoML local changes detected - Local BentoML repository including all code changes will be bundled together with the BentoService bundle. When used with docker, the base docker image will be default to same version as last PyPI release at version: 0.6.2. You can also force bentoml to use a specific version for deploying your BentoService bundle, by setting the config 'core/bentoml_deploy_version' to a pinned version or your custom BentoML on github, e.g.:'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
[2020-02-13 14:28:34,484] WARNING - Saved BentoService bundle version mismatch: loading BentoServie bundle create with BentoML version 0.6.2,  but loading from BentoML version 0.6.2+8.gd95a887
{
  "name": "PyTorchFashionClassifier",
  "version": "20200213142355_A51D6E",
  "created_at": "2020-02-13T22:24:21.765194Z",
  "env": {
    "conda_env": "name: bentoml-PyTorchFashionClassifier\nchannels:\n- defaults\ndependencies:\n- python=3.7.3\n- pip\n",
    "pip_dependencies": "bentoml==0.6.2\ntorch\nnumpy\ntorchvision\nscikit-learn\nimageio",
    "python_version": "3.7.3"
  },
  "artifacts": [
    {
      "name": "classifier",
      "artifact_type": "PytorchModelArtifact"
    }
  ],
  "apis": [
    {
      "name": "predict",
      "handler_type": "ImageHandler",
      "docs": "BentoService API",
      "handler_config": {
        "pilmode": "L",
        "input_names": [
          "image"
        ],
        "accept_image_formats": [
          ".jpg",
          ".png",
          ".jpeg",
          ".tiff",
          ".webp",
          ".bmp"
        ]
      }
    }
  ]
}
In [29]:
!PyTorchFashionClassifier run predict --input sample_image.png
[2020-02-13 14:29:14,734] WARNING - BentoML local changes detected - Local BentoML repository including all code changes will be bundled together with the BentoService bundle. When used with docker, the base docker image will be default to same version as last PyPI release at version: 0.6.2. You can also force bentoml to use a specific version for deploying your BentoService bundle, by setting the config 'core/bentoml_deploy_version' to a pinned version or your custom BentoML on github, e.g.:'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
[2020-02-13 14:29:14,745] WARNING - Saved BentoService bundle version mismatch: loading BentoServie bundle create with BentoML version 0.6.2,  but loading from BentoML version 0.6.2+8.gd95a887
[2020-02-13 14:29:14,748] WARNING - BentoML local changes detected - Local BentoML repository including all code changes will be bundled together with the BentoService bundle. When used with docker, the base docker image will be default to same version as last PyPI release at version: 0.6.2. You can also force bentoml to use a specific version for deploying your BentoService bundle, by setting the config 'core/bentoml_deploy_version' to a pinned version or your custom BentoML on github, e.g.:'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
[2020-02-13 14:29:14,752] WARNING - BentoML local changes detected - Local BentoML repository including all code changes will be bundled together with the BentoService bundle. When used with docker, the base docker image will be default to same version as last PyPI release at version: 0.6.2. You can also force bentoml to use a specific version for deploying your BentoService bundle, by setting the config 'core/bentoml_deploy_version' to a pinned version or your custom BentoML on github, e.g.:'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
/usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages/torch/nn/modules/container.py:92: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.
  input = module(input)
Bag

Containerize REST API server with Docker

** Make sure you have docker installed, note that it is not available when running in Google Colaboratory

In [21]:
!cd {saved_path} && docker build . -t pytorch-fashion-mnist
Sending build context to Docker daemon  133.6kB
Step 1/11 : FROM continuumio/miniconda3
 ---> ae46c364060f
Step 2/11 : ENTRYPOINT [ "/bin/bash", "-c" ]
 ---> Using cache
 ---> 2f135ada8e2d
Step 3/11 : EXPOSE 5000
 ---> Using cache
 ---> 738f652d09ae
Step 4/11 : RUN set -x      && apt-get update      && apt-get install --no-install-recommends --no-install-suggests -y libpq-dev build-essential      && rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> 70c62a45013a
Step 5/11 : RUN conda update conda -y       && conda install pip numpy scipy       && pip install gunicorn six
 ---> Using cache
 ---> fe5d966ecc35
Step 6/11 : COPY . /bento
 ---> d00c6768ebbe
Step 7/11 : WORKDIR /bento
 ---> Running in 65e4f1f8e11f
Removing intermediate container 65e4f1f8e11f
 ---> d0a6e7065765
Step 8/11 : RUN conda env update -n base -f /bento/environment.yml
 ---> Running in 6166f7e1de32
Collecting package metadata: ...working... done
Solving environment: ...working... 
The environment is inconsistent, please check the package plan carefully
The following packages are causing the inconsistency:

  - defaults/linux-64::pyopenssl==18.0.0=py37_0
  - defaults/linux-64::idna==2.8=py37_0
  - defaults/linux-64::six==1.12.0=py37_0
  - defaults/linux-64::cryptography==2.4.2=py37h1ba5d50_0
  - defaults/linux-64::ruamel_yaml==0.15.46=py37h14c3975_0
  - defaults/linux-64::pycparser==2.19=py37_0
  - defaults/linux-64::libedit==3.1.20170329=h6b74fdf_2
  - defaults/linux-64::zlib==1.2.11=h7b6447c_3
  - defaults/linux-64::readline==7.0=h7b6447c_5
  - defaults/linux-64::tk==8.6.8=hbc83047_0
  - defaults/linux-64::urllib3==1.24.1=py37_0
  - defaults/linux-64::pycosat==0.6.3=py37h14c3975_0
  - defaults/linux-64::ncurses==6.1=he6710b0_1
  - defaults/linux-64::requests==2.21.0=py37_0
  - defaults/linux-64::wheel==0.32.3=py37_0
  - defaults/linux-64::asn1crypto==0.24.0=py37_0
  - defaults/linux-64::chardet==3.0.4=py37_1
  - defaults/linux-64::setuptools==40.6.3=py37_0
  - defaults/linux-64::cffi==1.11.5=py37he75722e_1
  - defaults/linux-64::sqlite==3.26.0=h7b6447c_0
  - defaults/linux-64::xz==5.2.4=h14c3975_4
  - defaults/linux-64::libffi==3.2.1=hd88cf55_4
  - defaults/linux-64::libgcc-ng==8.2.0=hdf63c60_1
  - defaults/linux-64::yaml==0.1.7=had09818_2
  - defaults/linux-64::pysocks==1.6.8=py37_0
  - defaults/linux-64::python==3.7.1=h0371630_7
  - defaults/linux-64::pip==19.0.3=py37_0
  - defaults/linux-64::mkl_random==1.0.2=py37hd81dba3_0
  - defaults/linux-64::numpy-base==1.16.3=py37hde5b4d6_0
  - defaults/linux-64::mkl_fft==1.0.10=py37ha843d7b_0
  - defaults/linux-64::openssl==1.1.1b=h7b6447c_1
  - defaults/linux-64::conda==4.6.14=py37_0
  - defaults/linux-64::certifi==2019.3.9=py37_0
  - defaults/linux-64::scipy==1.2.1=py37h7c811a0_0
  - defaults/linux-64::numpy==1.16.3=py37h7e9f1db_0
done

Downloading and Extracting Packages
libedit-3.1.20181209 | 188 KB    | ########## | 100% 
sqlite-3.29.0        | 1.9 MB    | ########## | 100% 
certifi-2019.6.16    | 156 KB    | ########## | 100% 
ca-certificates-2019 | 134 KB    | ########## | 100% 
openssl-1.1.1d       | 3.7 MB    | ########## | 100% 
wheel-0.33.4         | 39 KB     | ########## | 100% 
libgcc-ng-9.1.0      | 8.1 MB    | ########## | 100% 
pip-19.2.2           | 1.9 MB    | ########## | 100% 
python-3.7.3         | 36.7 MB   | ########## | 100% 
_libgcc_mutex-0.1    | 3 KB      | ########## | 100% 
setuptools-41.0.1    | 648 KB    | ########## | 100% 
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done


==> WARNING: A newer version of conda exists. <==
  current version: 4.6.14
  latest version: 4.7.12

Please update conda by running

    $ conda update -n base -c defaults conda


Ran pip subprocess with arguments:
['/opt/conda/bin/python', '-m', 'pip', 'install', '-U', '-r', '/bento/condaenv.zv4o07od.requirements.txt']
Pip subprocess output:
Collecting bentoml[api_server]==0.4.1 (from -r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/45/7e/bef20c3d36ae7455c5ff3b7629c93b176df22f6131872857a3c746c031bb/BentoML-0.4.1-py3-none-any.whl (159kB)
Collecting flask (from bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/9b/93/628509b8d5dc749656a9641f4caf13540e2cdec85276964ff8f43bbb1d3b/Flask-1.1.1-py2.py3-none-any.whl (94kB)
Collecting ruamel.yaml>=0.15.0 (from bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/fa/90/ecff85a2e9c497e2fa7142496e10233556b5137db5bd46f3f3b006935ca8/ruamel.yaml-0.16.5-py2.py3-none-any.whl (123kB)
Collecting pathlib2 (from bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/67/c6/4dbf5dfdbe1140cadf765c3896acc098578626c35721bc7d3eb35f6a8fc1/pathlib2-2.3.4-py2.py3-none-any.whl
Collecting grpcio (from bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/e5/27/1f908ebb99c8d48a5ba4eb9d7997f5633b920d98fe712f67aaa0663f1307/grpcio-1.23.0-cp37-cp37m-manylinux1_x86_64.whl (2.2MB)
Collecting python-json-logger (from bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/80/9d/1c3393a6067716e04e6fcef95104c8426d262b4adaf18d7aa2470eab028d/python-json-logger-0.1.11.tar.gz
Collecting protobuf>=3.6.0 (from bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/30/fd/60ce148d8e4205bdf6da4ffec31348fd33f710c20a882b44319d54fd51ae/protobuf-3.9.1-cp37-cp37m-manylinux1_x86_64.whl (1.2MB)
Collecting prometheus-client (from bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/b3/23/41a5a24b502d35a4ad50a5bb7202a5e1d9a0364d0c12f56db3dbf7aca76d/prometheus_client-0.7.1.tar.gz
Collecting sqlalchemy (from bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/fc/49/82d64d705ced344ba458197dadab30cfa745f9650ee22260ac2b275d288c/SQLAlchemy-1.3.8.tar.gz (5.9MB)
Requirement already satisfied, skipping upgrade: gunicorn in /opt/conda/lib/python3.7/site-packages (from bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1)) (19.9.0)
Collecting dill (from bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/39/7a/70803635c850e351257029089d38748516a280864c97cbc73087afef6d51/dill-0.3.0.tar.gz (151kB)
Collecting pandas (from bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/7e/ab/ea76361f9d3e732e114adcd801d2820d5319c23d0ac5482fa3b412db217e/pandas-0.25.1-cp37-cp37m-manylinux1_x86_64.whl (10.4MB)
Collecting configparser (from bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/7a/2a/95ed0501cf5d8709490b1d3a3f9b5cf340da6c433f896bbe9ce08dbe6785/configparser-4.0.2-py2.py3-none-any.whl
Collecting boto3 (from bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/7a/2d/023459434edb679dea2c1547a75a5cb64d6c5c93ab601d664fde88c0bd83/boto3-1.9.231-py2.py3-none-any.whl (128kB)
Requirement already satisfied, skipping upgrade: numpy in /opt/conda/lib/python3.7/site-packages (from bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1)) (1.16.3)
Requirement already satisfied, skipping upgrade: six in /opt/conda/lib/python3.7/site-packages (from bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1)) (1.12.0)
Collecting click (from bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl (81kB)
Requirement already satisfied, skipping upgrade: requests in /opt/conda/lib/python3.7/site-packages (from bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1)) (2.21.0)
Collecting packaging (from bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/cf/94/9672c2d4b126e74c4496c6b3c58a8b51d6419267be9e70660ba23374c875/packaging-19.2-py2.py3-none-any.whl
Collecting cerberus (from bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/c9/0e/f78e23b778c2234972d364d0f8bea2de0a09f450f65d3f05ce091dd0f104/Cerberus-1.3.1.tar.gz (52kB)
Collecting docker (from bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/95/47/5560c9cf0c92b50da24216f0e7733250fbed5a497f69e3c70e1be62143fe/docker-4.0.2-py2.py3-none-any.whl (138kB)
Collecting Werkzeug; extra == "api_server" (from bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/ce/42/3aeda98f96e85fd26180534d36570e4d18108d62ae36f87694b476b83d6f/Werkzeug-0.16.0-py2.py3-none-any.whl (327kB)
Collecting Jinja2>=2.10.1 (from flask->bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/1d/e7/fd8b501e7a6dfe492a433deb7b9d833d39ca74916fa8bc63dd1a4947a671/Jinja2-2.10.1-py2.py3-none-any.whl (124kB)
Collecting itsdangerous>=0.24 (from flask->bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/76/ae/44b03b253d6fade317f32c24d100b3b35c2239807046a4c953c7b89fa49e/itsdangerous-1.1.0-py2.py3-none-any.whl
Collecting ruamel.yaml.clib>=0.1.2; platform_python_implementation == "CPython" and python_version < "3.8" (from ruamel.yaml>=0.15.0->bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/f2/bd/ff0d632e1b248c28427a23bd741f5b8f954acba82df86661e5daf7cc79a6/ruamel.yaml.clib-0.1.2-cp37-cp37m-manylinux1_x86_64.whl (547kB)
Requirement already satisfied, skipping upgrade: setuptools in /opt/conda/lib/python3.7/site-packages (from protobuf>=3.6.0->bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1)) (41.0.1)
Collecting pytz>=2017.2 (from pandas->bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/87/76/46d697698a143e05f77bec5a526bf4e56a0be61d63425b68f4ba553b51f2/pytz-2019.2-py2.py3-none-any.whl (508kB)
Collecting python-dateutil>=2.6.1 (from pandas->bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB)
Collecting botocore<1.13.0,>=1.12.231 (from boto3->bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/b1/18/90fa120b459ef691a19d2972790834b7eaf843e944d99d798005cb14e8fa/botocore-1.12.231-py2.py3-none-any.whl (5.7MB)
Collecting jmespath<1.0.0,>=0.7.1 (from boto3->bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl
Collecting s3transfer<0.3.0,>=0.2.0 (from boto3->bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/16/8a/1fc3dba0c4923c2a76e1ff0d52b305c44606da63f718d14d3231e21c51b0/s3transfer-0.2.1-py2.py3-none-any.whl (70kB)
Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1)) (3.0.4)
Requirement already satisfied, skipping upgrade: urllib3<1.25,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1)) (1.24.1)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1)) (2019.6.16)
Requirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1)) (2.8)
Collecting pyparsing>=2.0.2 (from packaging->bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/11/fa/0160cd525c62d7abd076a070ff02b2b94de589f1a9789774f17d7c54058e/pyparsing-2.4.2-py2.py3-none-any.whl (65kB)
Collecting websocket-client>=0.32.0 (from docker->bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/29/19/44753eab1fdb50770ac69605527e8859468f3c0fd7dc5a76dd9c4dbd7906/websocket_client-0.56.0-py2.py3-none-any.whl (200kB)
Collecting MarkupSafe>=0.23 (from Jinja2>=2.10.1->flask->bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/98/7b/ff284bd8c80654e471b769062a9b43cc5d03e7a615048d96f4619df8d420/MarkupSafe-1.1.1-cp37-cp37m-manylinux1_x86_64.whl
Collecting docutils<0.16,>=0.10 (from botocore<1.13.0,>=1.12.231->boto3->bentoml[api_server]==0.4.1->-r /bento/condaenv.zv4o07od.requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/22/cd/a6aa959dca619918ccb55023b4cb151949c64d4d5d55b3f4ffd7eee0c6e8/docutils-0.15.2-py3-none-any.whl (547kB)
Building wheels for collected packages: python-json-logger, prometheus-client, sqlalchemy, dill, cerberus
  Building wheel for python-json-logger (setup.py): started
  Building wheel for python-json-logger (setup.py): finished with status 'done'
  Created wheel for python-json-logger: filename=python_json_logger-0.1.11-py2.py3-none-any.whl size=5077 sha256=b6be66e346371d286ecd9e89c98867aeda49c57e978779718f8f7e6b0ececf83
  Stored in directory: /root/.cache/pip/wheels/97/f7/a1/752e22bb30c1cfe38194ea0070a5c66e76ef4d06ad0c7dc401
  Building wheel for prometheus-client (setup.py): started
  Building wheel for prometheus-client (setup.py): finished with status 'done'
  Created wheel for prometheus-client: filename=prometheus_client-0.7.1-cp37-none-any.whl size=41405 sha256=c70c6f2c47a9daff675335b505dbb9056b504fe915d6db3f376ef20f4cc8f8c7
  Stored in directory: /root/.cache/pip/wheels/1c/54/34/fd47cd9b308826cc4292b54449c1899a30251ef3b506bc91ea
  Building wheel for sqlalchemy (setup.py): started
  Building wheel for sqlalchemy (setup.py): finished with status 'done'
  Created wheel for sqlalchemy: filename=SQLAlchemy-1.3.8-cp37-cp37m-linux_x86_64.whl size=1197985 sha256=298499650bcbceae0dda2404f393a0709ea18495e8d137e1554b5ef02bfcc562
  Stored in directory: /root/.cache/pip/wheels/97/b6/66/de2064d40c920adc2984ff3b8fd4f11494c8ab9e48ba87e8a2
  Building wheel for dill (setup.py): started
  Building wheel for dill (setup.py): finished with status 'done'
  Created wheel for dill: filename=dill-0.3.0-cp37-none-any.whl size=77512 sha256=1388231f916366300a3449c95674a3a96979637333384eafae9bf179591af69d
  Stored in directory: /root/.cache/pip/wheels/c9/de/a4/a91eec4eea652104d8c81b633f32ead5eb57d1b294eab24167
  Building wheel for cerberus (setup.py): started
  Building wheel for cerberus (setup.py): finished with status 'done'
  Created wheel for cerberus: filename=Cerberus-1.3.1-cp37-none-any.whl size=54129 sha256=8788b3c5839d1466149ca68564a94af3043a447be2e96e1d0d9f876652cbdcfb
  Stored in directory: /root/.cache/pip/wheels/4f/b4/2b/83fc9a02bdbb49350a9eeb6d34753e4d7df7f77cb027e1d4b3
Successfully built python-json-logger prometheus-client sqlalchemy dill cerberus
Installing collected packages: MarkupSafe, Jinja2, click, Werkzeug, itsdangerous, flask, ruamel.yaml.clib, ruamel.yaml, pathlib2, grpcio, python-json-logger, protobuf, prometheus-client, sqlalchemy, dill, pytz, python-dateutil, pandas, configparser, jmespath, docutils, botocore, s3transfer, boto3, pyparsing, packaging, cerberus, websocket-client, docker, bentoml
Successfully installed Jinja2-2.10.1 MarkupSafe-1.1.1 Werkzeug-0.16.0 bentoml-0.4.1 boto3-1.9.231 botocore-1.12.231 cerberus-1.3.1 click-7.0 configparser-4.0.2 dill-0.3.0 docker-4.0.2 docutils-0.15.2 flask-1.1.1 grpcio-1.23.0 itsdangerous-1.1.0 jmespath-0.9.4 packaging-19.2 pandas-0.25.1 pathlib2-2.3.4 prometheus-client-0.7.1 protobuf-3.9.1 pyparsing-2.4.2 python-dateutil-2.8.0 python-json-logger-0.1.11 pytz-2019.2 ruamel.yaml-0.16.5 ruamel.yaml.clib-0.1.2 s3transfer-0.2.1 sqlalchemy-1.3.8 websocket-client-0.56.0

#
# To activate this environment, use:
# > conda activate base
#
# To deactivate an active environment, use:
# > conda deactivate
#

Removing intermediate container 6166f7e1de32
 ---> 666f7c81f870
Step 9/11 : RUN pip install -r /bento/requirements.txt
 ---> Running in 85f87d646259
Requirement already satisfied: bentoml==0.4.1 in /opt/conda/lib/python3.7/site-packages (from -r /bento/requirements.txt (line 1)) (0.4.1)
Collecting torch (from -r /bento/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/05/65/5248be50c55ab7429dd5c11f5e2f9f5865606b80e854ca63139ad1a584f2/torch-1.2.0-cp37-cp37m-manylinux1_x86_64.whl (748.9MB)
Requirement already satisfied: numpy in /opt/conda/lib/python3.7/site-packages (from -r /bento/requirements.txt (line 3)) (1.16.3)
Collecting torchvision (from -r /bento/requirements.txt (line 4))
  Downloading https://files.pythonhosted.org/packages/51/83/2d77d040e34bd8f70dcb4770f7eb7d0aa71e07738abf6831be863ade00db/torchvision-0.4.0-cp37-cp37m-manylinux1_x86_64.whl (8.8MB)
Collecting scikit-learn (from -r /bento/requirements.txt (line 5))
  Downloading https://files.pythonhosted.org/packages/9f/c5/e5267eb84994e9a92a2c6a6ee768514f255d036f3c8378acfa694e9f2c99/scikit_learn-0.21.3-cp37-cp37m-manylinux1_x86_64.whl (6.7MB)
Collecting imageio (from -r /bento/requirements.txt (line 6))
  Downloading https://files.pythonhosted.org/packages/af/0a/943c965d372dae0b1f1482677d29030ab834351a61a9a632fd62f27f1523/imageio-2.5.0-py3-none-any.whl (3.3MB)
Requirement already satisfied: flask in /opt/conda/lib/python3.7/site-packages (from bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (1.1.1)
Requirement already satisfied: pathlib2 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (2.3.4)
Requirement already satisfied: configparser in /opt/conda/lib/python3.7/site-packages (from bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (4.0.2)
Requirement already satisfied: python-json-logger in /opt/conda/lib/python3.7/site-packages (from bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (0.1.11)
Requirement already satisfied: grpcio in /opt/conda/lib/python3.7/site-packages (from bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (1.23.0)
Requirement already satisfied: click in /opt/conda/lib/python3.7/site-packages (from bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (7.0)
Requirement already satisfied: protobuf>=3.6.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (3.9.1)
Requirement already satisfied: six in /opt/conda/lib/python3.7/site-packages (from bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (1.12.0)
Requirement already satisfied: ruamel.yaml>=0.15.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (0.16.5)
Requirement already satisfied: packaging in /opt/conda/lib/python3.7/site-packages (from bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (19.2)
Requirement already satisfied: cerberus in /opt/conda/lib/python3.7/site-packages (from bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (1.3.1)
Requirement already satisfied: requests in /opt/conda/lib/python3.7/site-packages (from bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (2.21.0)
Requirement already satisfied: boto3 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (1.9.231)
Requirement already satisfied: dill in /opt/conda/lib/python3.7/site-packages (from bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (0.3.0)
Requirement already satisfied: gunicorn in /opt/conda/lib/python3.7/site-packages (from bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (19.9.0)
Requirement already satisfied: sqlalchemy in /opt/conda/lib/python3.7/site-packages (from bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (1.3.8)
Requirement already satisfied: prometheus-client in /opt/conda/lib/python3.7/site-packages (from bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (0.7.1)
Requirement already satisfied: pandas in /opt/conda/lib/python3.7/site-packages (from bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (0.25.1)
Requirement already satisfied: docker in /opt/conda/lib/python3.7/site-packages (from bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (4.0.2)
Collecting pillow>=4.1.1 (from torchvision->-r /bento/requirements.txt (line 4))
  Downloading https://files.pythonhosted.org/packages/a4/da/2bd281c875686230eabc13d20ab590ea617563b0e746abfb0698c4d5b645/Pillow-6.1.0-cp37-cp37m-manylinux1_x86_64.whl (2.1MB)
Requirement already satisfied: scipy>=0.17.0 in /opt/conda/lib/python3.7/site-packages (from scikit-learn->-r /bento/requirements.txt (line 5)) (1.2.1)
Collecting joblib>=0.11 (from scikit-learn->-r /bento/requirements.txt (line 5))
  Downloading https://files.pythonhosted.org/packages/cd/c1/50a758e8247561e58cb87305b1e90b171b8c767b15b12a1734001f41d356/joblib-0.13.2-py2.py3-none-any.whl (278kB)
Requirement already satisfied: Werkzeug>=0.15 in /opt/conda/lib/python3.7/site-packages (from flask->bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (0.16.0)
Requirement already satisfied: Jinja2>=2.10.1 in /opt/conda/lib/python3.7/site-packages (from flask->bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (2.10.1)
Requirement already satisfied: itsdangerous>=0.24 in /opt/conda/lib/python3.7/site-packages (from flask->bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (1.1.0)
Requirement already satisfied: setuptools in /opt/conda/lib/python3.7/site-packages (from protobuf>=3.6.0->bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (41.0.1)
Requirement already satisfied: ruamel.yaml.clib>=0.1.2; platform_python_implementation == "CPython" and python_version < "3.8" in /opt/conda/lib/python3.7/site-packages (from ruamel.yaml>=0.15.0->bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (0.1.2)
Requirement already satisfied: pyparsing>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from packaging->bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (2.4.2)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (2019.6.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (3.0.4)
Requirement already satisfied: urllib3<1.25,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (1.24.1)
Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (2.8)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /opt/conda/lib/python3.7/site-packages (from boto3->bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (0.9.4)
Requirement already satisfied: s3transfer<0.3.0,>=0.2.0 in /opt/conda/lib/python3.7/site-packages (from boto3->bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (0.2.1)
Requirement already satisfied: botocore<1.13.0,>=1.12.231 in /opt/conda/lib/python3.7/site-packages (from boto3->bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (1.12.231)
Requirement already satisfied: python-dateutil>=2.6.1 in /opt/conda/lib/python3.7/site-packages (from pandas->bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (2.8.0)
Requirement already satisfied: pytz>=2017.2 in /opt/conda/lib/python3.7/site-packages (from pandas->bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (2019.2)
Requirement already satisfied: websocket-client>=0.32.0 in /opt/conda/lib/python3.7/site-packages (from docker->bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (0.56.0)
Requirement already satisfied: MarkupSafe>=0.23 in /opt/conda/lib/python3.7/site-packages (from Jinja2>=2.10.1->flask->bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (1.1.1)
Requirement already satisfied: docutils<0.16,>=0.10 in /opt/conda/lib/python3.7/site-packages (from botocore<1.13.0,>=1.12.231->boto3->bentoml==0.4.1->-r /bento/requirements.txt (line 1)) (0.15.2)
Installing collected packages: torch, pillow, torchvision, joblib, scikit-learn, imageio
Successfully installed imageio-2.5.0 joblib-0.13.2 pillow-6.1.0 scikit-learn-0.21.3 torch-1.2.0 torchvision-0.4.0
Removing intermediate container 85f87d646259
 ---> b4f9c1c9445b
Step 10/11 : RUN if [ -f /bento/setup.sh ]; then /bin/bash -c /bento/setup.sh; fi
 ---> Running in 71825c516006
Removing intermediate container 71825c516006
 ---> bc37731d5d39
Step 11/11 : CMD ["bentoml serve-gunicorn /bento"]
 ---> Running in 4050988df254
Removing intermediate container 4050988df254
 ---> d6ed6efbdf93
Successfully built d6ed6efbdf93
Successfully tagged pytorch-fashion-mnist:latest
In [22]:
!docker run -p 5000:5000 pytorch-fashion-mnist
[2019-09-19 19:57:42 +0000] [1] [INFO] Starting gunicorn 19.9.0
[2019-09-19 19:57:42 +0000] [1] [INFO] Listening at: http://0.0.0.0:5000 (1)
[2019-09-19 19:57:42 +0000] [1] [INFO] Using worker: sync
[2019-09-19 19:57:42 +0000] [10] [INFO] Booting worker with pid: 10
[2019-09-19 19:57:42 +0000] [11] [INFO] Booting worker with pid: 11
[2019-09-19 19:57:42 +0000] [12] [INFO] Booting worker with pid: 12
^C
[2019-09-19 20:04:03 +0000] [1] [INFO] Handling signal: int
/opt/conda/lib/python3.7/site-packages/torch/nn/modules/container.py:92: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.
  input = module(input)
[2019-09-19 20:04:03 +0000] [12] [INFO] Worker exiting (pid: 12)
[2019-09-19 20:04:03 +0000] [10] [INFO] Worker exiting (pid: 10)
[2019-09-19 20:04:03 +0000] [11] [INFO] Worker exiting (pid: 11)

Deploy BentoService as REST API server to the cloud

BentoML support deployment to multiply cloud provider services, such as AWS Lambda, AWS Sagemaker, Google Cloudrun and etc. You can find the full list and guide on the documentation site at https://docs.bentoml.org/en/latest/deployment/index.html

For this project, we are going to deploy to AWS Sagemaker

Deploy to Sagemaker with single command bentoml sagemaker deploy

In [30]:
!bentoml sagemaker deploy pytorch-fashion -b PyTorchFashionClassifier:20200213142355_A51D6E --api-name predict
Deploying Sagemaker deployment \[2020-02-13 14:31:00,094] INFO - Step 1/11 : FROM continuumio/miniconda3:4.7.12
[2020-02-13 14:31:00,095] INFO - 

[2020-02-13 14:31:00,095] INFO -  ---> 406f2b43ea59

[2020-02-13 14:31:00,095] INFO - Step 2/11 : EXPOSE 8080
[2020-02-13 14:31:00,095] INFO - 

[2020-02-13 14:31:00,096] INFO -  ---> Using cache

[2020-02-13 14:31:00,096] INFO -  ---> 85b4bb5fff81

[2020-02-13 14:31:00,096] INFO - Step 3/11 : RUN set -x      && apt-get update      && apt-get install --no-install-recommends --no-install-suggests -y libpq-dev build-essential     && apt-get install -y nginx      && rm -rf /var/lib/apt/lists/*
[2020-02-13 14:31:00,096] INFO - 

[2020-02-13 14:31:00,096] INFO -  ---> Using cache

[2020-02-13 14:31:00,096] INFO -  ---> 66280bc479d1

[2020-02-13 14:31:00,096] INFO - Step 4/11 : RUN conda install pip numpy scipy       && pip install gunicorn gevent
[2020-02-13 14:31:00,097] INFO - 

[2020-02-13 14:31:00,097] INFO -  ---> Using cache

[2020-02-13 14:31:00,097] INFO -  ---> 1dbfc408f11d

[2020-02-13 14:31:00,097] INFO - Step 5/11 : COPY . /opt/program
[2020-02-13 14:31:00,097] INFO - 

\[2020-02-13 14:31:00,551] INFO -  ---> 3c6e215c3dc8

[2020-02-13 14:31:00,551] INFO - Step 6/11 : WORKDIR /opt/program
[2020-02-13 14:31:00,552] INFO - 

/[2020-02-13 14:31:00,735] INFO -  ---> Running in 52d615890214

-[2020-02-13 14:31:01,027] INFO -  ---> 5e43ae14fe08

[2020-02-13 14:31:01,027] INFO - Step 7/11 : RUN conda env update -n base -f /opt/program/environment.yml
[2020-02-13 14:31:01,027] INFO - 

/[2020-02-13 14:31:01,154] INFO -  ---> Running in c06ec52df534

|[2020-02-13 14:31:02,890] INFO - Collecting package metadata (repodata.json): ...working... 
\[2020-02-13 14:31:08,267] INFO - done
Solving environment: ...working... 
|[2020-02-13 14:31:13,049] INFO - done

[2020-02-13 14:31:13,134] INFO - 
Downloading and Extracting Packages
python-3.7.3         | 32.1 MB   |            |   0% 
python-3.7.3         | 32.1 MB   |            |   0% 
python-3.7.3         | 32.1 MB   | 1          |   1% 
python-3.7.3         | 32.1 MB   | 5          |   6% 
python-3.7.3         | 32.1 MB   | 7          |   8% 
python-3.7.3         | 32.1 MB   | #          |  10% 
python-3.7.3         | 32.1 MB   | #1         |  12% 
python-3.7.3         | 32.1 MB   | #3         |  14% 
python-3.7.3         | 32.1 MB   | #5         |  15% 
python-3.7.3         | 32.1 MB   | #7         |  18% 
python-3.7.3         | 32.1 MB   | #9         |  19% 
python-3.7.3         | 32.1 MB   | ##1        |  21% 
python-3.7.3         | 32.1 MB   | ##2        |  23% 
python-3.7.3         | 32.1 MB   | ##3        |  24% 
python-3.7.3         | 32.1 MB   | ##5        |  25% 
python-3.7.3         | 32.1 MB   | ##6        |  26% 
python-3.7.3         | 32.1 MB   | ##8        |  28% 
python-3.7.3         | 32.1 MB   | ###        |  30% 
python-3.7.3         | 32.1 MB   | ###1       |  32% 
python-3.7.3         | 32.1 MB   | ###3       |  33% 
python-3.7.3         | 32.1 MB   | ###5       |  35% 
python-3.7.3         | 32.1 MB   | ###7       |  38% 
python-3.7.3         | 32.1 MB   | ###9       |  39% 
python-3.7.3         | 32.1 MB   | ####1      |  41% 
python-3.7.3         | 32.1 MB   | ####2      |  42% 
python-3.7.3         | 32.1 MB   | ####3      |  44% 
python-3.7.3         | 32.1 MB   | ####5      |  45% 
python-3.7.3         | 32.1 MB   | ####6      |  47% 
python-3.7.3         | 32.1 MB   | ####8      |  48% 
python-3.7.3         | 32.1 MB   | ####9      |  50% 
python-3.7.3         | 32.1 MB   | #####1     |  52% 
python-3.7.3         | 32.1 MB   | #####3     |  53% 
python-3.7.3         | 32.1 MB   | #####5     |  55% 
python-3.7.3         | 32.1 MB   | #####7     |  57% 
python-3.7.3         | 32.1 MB   | #####9     |  59% 
python-3.7.3         | 32.1 MB   | ######1    |  61% 
python-3.7.3         | 32.1 MB   | ######3    |  63% 
python-3.7.3         | 32.1 MB   | ######5    |  65% 
python-3.7.3         | 32.1 MB   | ######7    |  67% 
python-3.7.3         | 32.1 MB   | ######8    |  69% 
python-3.7.3         | 32.1 MB   | #######    |  71% 
python-3.7.3         | 32.1 MB   | #######2   |  72% 
python-3.7.3         | 32.1 MB   | #######4   |  74% 
python-3.7.3         | 32.1 MB   | #######6   |  76% 
python-3.7.3         | 32.1 MB   | #######7   |  78% 
python-3.7.3         | 32.1 MB   | #######9   |  80% 
python-3.7.3         | 32.1 MB   | ########1  |  82% 
python-3.7.3         | 32.1 MB   | ########3  |  83% 
python-3.7.3         | 32.1 MB   | ########4  |  85% 
python-3.7.3         | 32.1 MB   | ########7  |  87% 
python-3.7.3         | 32.1 MB   | ########8  |  89% 
python-3.7.3         | 32.1 MB   | #########  |  91% 
python-3.7.3         | 32.1 MB   | #########2 |  92% 
python-3.7.3         | 32.1 MB   | #########4 |  95% 
python-3.7.3         | 32.1 MB   | #########6 |  96% 
python-3.7.3         | 32.1 MB   | #########7 |  97% 
python-3.7.3         | 32.1 MB   | #########8 |  98% 
python-3.7.3         | 32.1 MB   | #########9 | 100% 
python-3.7.3         | 32.1 MB   | ########## | 100% 
[2020-02-13 14:31:21,815] INFO - 
Preparing transaction: 
[2020-02-13 14:31:21,815] INFO - ...working... 
/[2020-02-13 14:31:21,963] INFO - done

[2020-02-13 14:31:21,963] INFO - Verifying transaction: 
[2020-02-13 14:31:21,964] INFO - ...working... 
-[2020-02-13 14:31:22,731] INFO - done
Executing transaction: ...working... 
\[2020-02-13 14:31:26,670] INFO - done

/[2020-02-13 14:31:27,326] INFO - #
# To activate this environment, use
#
#     $ conda activate base
#
# To deactivate an active environment, use
#
#     $ conda deactivate


-[2020-02-13 14:31:30,473] INFO -  ---> 190cac8cf4e2

[2020-02-13 14:31:30,474] INFO - Step 8/11 : RUN pip install -r /opt/program/requirements.txt
[2020-02-13 14:31:30,474] INFO - 

|[2020-02-13 14:31:30,681] INFO -  ---> Running in 83227fbbb2e5

-[2020-02-13 14:31:32,106] INFO - Collecting bentoml==0.6.2

/[2020-02-13 14:31:32,189] INFO -   Downloading BentoML-0.6.2-py3-none-any.whl (554 kB)

/[2020-02-13 14:31:32,648] INFO - Collecting torch

[2020-02-13 14:31:32,675] INFO -   Downloading torch-1.4.0-cp37-cp37m-manylinux1_x86_64.whl (753.4 MB)

\[2020-02-13 14:34:37,513] INFO - Requirement already satisfied: numpy in /opt/conda/lib/python3.7/site-packages (from -r /opt/program/requirements.txt (line 3)) (1.18.1)

|[2020-02-13 14:34:39,002] INFO - Collecting torchvision

[2020-02-13 14:34:39,035] INFO -   Downloading torchvision-0.5.0-cp37-cp37m-manylinux1_x86_64.whl (4.0 MB)

|[2020-02-13 14:34:40,280] INFO - Collecting scikit-learn

[2020-02-13 14:34:40,294] INFO -   Downloading scikit_learn-0.22.1-cp37-cp37m-manylinux1_x86_64.whl (7.0 MB)

|[2020-02-13 14:34:42,684] INFO - Collecting imageio

[2020-02-13 14:34:42,694] INFO -   Downloading imageio-2.6.1-py3-none-any.whl (3.3 MB)

-[2020-02-13 14:34:43,307] INFO - Collecting flask

[2020-02-13 14:34:43,321] INFO -   Downloading Flask-1.1.1-py2.py3-none-any.whl (94 kB)

/[2020-02-13 14:34:43,399] INFO - Collecting cerberus

[2020-02-13 14:34:43,410] INFO -   Downloading Cerberus-1.3.2.tar.gz (52 kB)

-[2020-02-13 14:34:44,088] INFO - Collecting packaging

[2020-02-13 14:34:44,101] INFO -   Downloading packaging-20.1-py2.py3-none-any.whl (36 kB)

[2020-02-13 14:34:44,124] INFO - Requirement already satisfied: requests in /opt/conda/lib/python3.7/site-packages (from bentoml==0.6.2->-r /opt/program/requirements.txt (line 1)) (2.22.0)

[2020-02-13 14:34:44,147] INFO - Requirement already satisfied: gunicorn in /opt/conda/lib/python3.7/site-packages (from bentoml==0.6.2->-r /opt/program/requirements.txt (line 1)) (20.0.4)

/[2020-02-13 14:34:44,242] INFO - Collecting humanfriendly

[2020-02-13 14:34:44,253] INFO -   Downloading humanfriendly-6.1-py2.py3-none-any.whl (75 kB)

/[2020-02-13 14:34:45,446] INFO - Collecting pandas

[2020-02-13 14:34:45,475] INFO -   Downloading pandas-1.0.1-cp37-cp37m-manylinux1_x86_64.whl (10.1 MB)

/[2020-02-13 14:34:47,966] INFO - Collecting docker

[2020-02-13 14:34:47,978] INFO -   Downloading docker-4.2.0-py2.py3-none-any.whl (143 kB)

|[2020-02-13 14:34:48,071] INFO - Collecting configparser

[2020-02-13 14:34:48,080] INFO -   Downloading configparser-4.0.2-py2.py3-none-any.whl (22 kB)

/[2020-02-13 14:34:48,724] INFO - Collecting boto3

[2020-02-13 14:34:48,752] INFO -   Downloading boto3-1.11.17-py2.py3-none-any.whl (128 kB)

|[2020-02-13 14:34:48,826] INFO - Collecting prometheus-client

[2020-02-13 14:34:48,839] INFO -   Downloading prometheus_client-0.7.1.tar.gz (38 kB)

/[2020-02-13 14:34:49,162] INFO - Collecting python-dateutil<2.8.1,>=2.1

[2020-02-13 14:34:49,173] INFO -   Downloading python_dateutil-2.8.0-py2.py3-none-any.whl (226 kB)

|[2020-02-13 14:34:49,319] INFO - Collecting alembic

\[2020-02-13 14:34:49,338] INFO -   Downloading alembic-1.4.0.tar.gz (1.1 MB)

/[2020-02-13 14:34:50,365] INFO - Collecting protobuf>=3.6.0

[2020-02-13 14:34:50,376] INFO -   Downloading protobuf-3.11.3-cp37-cp37m-manylinux1_x86_64.whl (1.3 MB)

/[2020-02-13 14:34:50,784] INFO - Collecting click>=7.0

[2020-02-13 14:34:50,797] INFO -   Downloading Click-7.0-py2.py3-none-any.whl (81 kB)

/[2020-02-13 14:34:51,682] INFO - Collecting grpcio

|[2020-02-13 14:34:51,700] INFO -   Downloading grpcio-1.27.1-cp37-cp37m-manylinux2010_x86_64.whl (2.7 MB)

\[2020-02-13 14:34:53,118] INFO - Collecting ruamel.yaml>=0.15.0

-[2020-02-13 14:34:53,133] INFO -   Downloading ruamel.yaml-0.16.10-py2.py3-none-any.whl (111 kB)

|[2020-02-13 14:34:53,376] INFO - Collecting sqlalchemy>=1.3.0

[2020-02-13 14:34:53,385] INFO -   Downloading SQLAlchemy-1.3.13.tar.gz (6.0 MB)

|[2020-02-13 14:34:56,215] INFO - Collecting python-json-logger

[2020-02-13 14:34:56,226] INFO -   Downloading python-json-logger-0.1.11.tar.gz (6.0 kB)

-[2020-02-13 14:34:56,501] INFO - Collecting tabulate

/[2020-02-13 14:34:56,512] INFO -   Downloading tabulate-0.8.6.tar.gz (45 kB)

/[2020-02-13 14:34:57,339] INFO - Collecting pillow>=4.1.1

[2020-02-13 14:34:57,358] INFO -   Downloading Pillow-7.0.0-cp37-cp37m-manylinux1_x86_64.whl (2.1 MB)

|[2020-02-13 14:34:57,879] INFO - Requirement already satisfied: six in /opt/conda/lib/python3.7/site-packages (from torchvision->-r /opt/program/requirements.txt (line 4)) (1.12.0)

\[2020-02-13 14:34:58,009] INFO - Collecting joblib>=0.11

[2020-02-13 14:34:58,035] INFO -   Downloading joblib-0.14.1-py2.py3-none-any.whl (294 kB)

|[2020-02-13 14:34:58,328] INFO - Requirement already satisfied: scipy>=0.17.0 in /opt/conda/lib/python3.7/site-packages (from scikit-learn->-r /opt/program/requirements.txt (line 5)) (1.4.1)

\[2020-02-13 14:34:58,385] INFO - Collecting Jinja2>=2.10.1

[2020-02-13 14:34:58,407] INFO -   Downloading Jinja2-2.11.1-py2.py3-none-any.whl (126 kB)

-[2020-02-13 14:34:58,513] INFO - Collecting itsdangerous>=0.24

[2020-02-13 14:34:58,525] INFO -   Downloading itsdangerous-1.1.0-py2.py3-none-any.whl (16 kB)

/[2020-02-13 14:34:58,626] INFO - Collecting Werkzeug>=0.15

[2020-02-13 14:34:58,637] INFO -   Downloading Werkzeug-1.0.0-py2.py3-none-any.whl (298 kB)

\[2020-02-13 14:34:58,771] INFO - Requirement already satisfied: setuptools in /opt/conda/lib/python3.7/site-packages (from cerberus->bentoml==0.6.2->-r /opt/program/requirements.txt (line 1)) (41.4.0)

-[2020-02-13 14:34:58,960] INFO - Collecting pyparsing>=2.0.2

/[2020-02-13 14:34:58,974] INFO -   Downloading pyparsing-2.4.6-py2.py3-none-any.whl (67 kB)

[2020-02-13 14:34:59,013] INFO - Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.6.2->-r /opt/program/requirements.txt (line 1)) (2.8)

[2020-02-13 14:34:59,016] INFO - Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.6.2->-r /opt/program/requirements.txt (line 1)) (2019.11.28)

[2020-02-13 14:34:59,019] INFO - Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.6.2->-r /opt/program/requirements.txt (line 1)) (3.0.4)

[2020-02-13 14:34:59,023] INFO - Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.6.2->-r /opt/program/requirements.txt (line 1)) (1.24.2)

\[2020-02-13 14:34:59,246] INFO - Collecting pytz>=2017.2

[2020-02-13 14:34:59,258] INFO -   Downloading pytz-2019.3-py2.py3-none-any.whl (509 kB)

\[2020-02-13 14:34:59,590] INFO - Collecting websocket-client>=0.32.0

[2020-02-13 14:34:59,601] INFO -   Downloading websocket_client-0.57.0-py2.py3-none-any.whl (200 kB)

-[2020-02-13 14:34:59,701] INFO - Collecting jmespath<1.0.0,>=0.7.1

[2020-02-13 14:34:59,714] INFO -   Downloading jmespath-0.9.4-py2.py3-none-any.whl (24 kB)

|[2020-02-13 14:35:00,299] INFO - Collecting botocore<1.15.0,>=1.14.17

[2020-02-13 14:35:00,318] INFO -   Downloading botocore-1.14.17-py2.py3-none-any.whl (5.9 MB)

\[2020-02-13 14:35:02,120] INFO - Collecting s3transfer<0.4.0,>=0.3.0

[2020-02-13 14:35:02,134] INFO -   Downloading s3transfer-0.3.3-py2.py3-none-any.whl (69 kB)

-[2020-02-13 14:35:02,218] INFO - Collecting Mako

[2020-02-13 14:35:02,234] INFO -   Downloading Mako-1.1.1.tar.gz (468 kB)

/[2020-02-13 14:35:02,722] INFO - Collecting python-editor>=0.3

[2020-02-13 14:35:02,737] INFO -   Downloading python_editor-1.0.4-py3-none-any.whl (4.9 kB)

|[2020-02-13 14:35:02,802] INFO - Collecting ruamel.yaml.clib>=0.1.2; platform_python_implementation == "CPython" and python_version < "3.9"

[2020-02-13 14:35:02,824] INFO -   Downloading ruamel.yaml.clib-0.2.0-cp37-cp37m-manylinux1_x86_64.whl (547 kB)

/[2020-02-13 14:35:03,072] INFO - Collecting MarkupSafe>=0.23

[2020-02-13 14:35:03,088] INFO -   Downloading MarkupSafe-1.1.1-cp37-cp37m-manylinux1_x86_64.whl (27 kB)

[2020-02-13 14:35:03,143] INFO - Collecting docutils<0.16,>=0.10

[2020-02-13 14:35:03,157] INFO -   Downloading docutils-0.15.2-py3-none-any.whl (547 kB)

\[2020-02-13 14:35:03,333] INFO - Building wheels for collected packages: cerberus, prometheus-client, alembic, sqlalchemy, python-json-logger, tabulate, Mako

[2020-02-13 14:35:03,334] INFO -   Building wheel for cerberus (setup.py): started

|[2020-02-13 14:35:03,654] INFO -   Building wheel for cerberus (setup.py): finished with status 'done'

[2020-02-13 14:35:03,661] INFO -   Created wheel for cerberus: filename=Cerberus-1.3.2-py3-none-any.whl size=54335 sha256=f702d0a95aaa3d389b27ba5e86ac0b8354fbaee4399da2804e2a45b6f3eaf3e3
  Stored in directory: /root/.cache/pip/wheels/17/3a/0d/e2fc48cf85cb858f5e65f1baa36180ebb5dce6397c35c4cfcb

[2020-02-13 14:35:03,662] INFO -   Building wheel for prometheus-client (setup.py): started

/[2020-02-13 14:35:03,956] INFO -   Building wheel for prometheus-client (setup.py): finished with status 'done'

[2020-02-13 14:35:03,956] INFO -   Created wheel for prometheus-client: filename=prometheus_client-0.7.1-py3-none-any.whl size=41402 sha256=86fb50e126740340c291fffd58a91a6be7a6051c2d750a0a3b40055e2a2fd480
  Stored in directory: /root/.cache/pip/wheels/30/0c/26/59ba285bf65dc79d195e9b25e2ddde4c61070422729b0cd914

[2020-02-13 14:35:03,959] INFO -   Building wheel for alembic (setup.py): started

/[2020-02-13 14:35:04,339] INFO -   Building wheel for alembic (setup.py): finished with status 'done'

[2020-02-13 14:35:04,340] INFO -   Created wheel for alembic: filename=alembic-1.4.0-py2.py3-none-any.whl size=157563 sha256=37426f5a1a6b0bce2282f78ae0b2e5ed5054f982bc02d83b6b993749d8c83a47
  Stored in directory: /root/.cache/pip/wheels/33/a9/f9/a53f885636269db5b76cf7afa3a1ab86d9d2fe96610d09274e

[2020-02-13 14:35:04,342] INFO -   Building wheel for sqlalchemy (setup.py): started

/[2020-02-13 14:35:05,974] INFO -   Building wheel for sqlalchemy (setup.py): finished with status 'done'

[2020-02-13 14:35:05,980] INFO -   Created wheel for sqlalchemy: filename=SQLAlchemy-1.3.13-cp37-cp37m-linux_x86_64.whl size=1223686 sha256=4e8babce2bbdccc02b8496265d9d815c3079deb3bd524f536cd7a864f31c3f9b
  Stored in directory: /root/.cache/pip/wheels/b9/ba/77/163f10f14bd489351530603e750c195b0ceceed2f3be2b32f1

[2020-02-13 14:35:05,981] INFO -   Building wheel for python-json-logger (setup.py): started

-[2020-02-13 14:35:06,247] INFO -   Building wheel for python-json-logger (setup.py): finished with status 'done'

[2020-02-13 14:35:06,248] INFO -   Created wheel for python-json-logger: filename=python_json_logger-0.1.11-py2.py3-none-any.whl size=5076 sha256=cf4c02fd3273e7cfbef2b94d0308f84d2f74aa80636d5250d07edfffe48bb46e
  Stored in directory: /root/.cache/pip/wheels/fa/7f/fd/92ccdbb9d1a65486406e0363d2ba5b4ce52f400a915f602ecb

[2020-02-13 14:35:06,250] INFO -   Building wheel for tabulate (setup.py): started

\[2020-02-13 14:35:06,535] INFO -   Building wheel for tabulate (setup.py): finished with status 'done'

[2020-02-13 14:35:06,536] INFO -   Created wheel for tabulate: filename=tabulate-0.8.6-py3-none-any.whl size=23273 sha256=f69d7a2c0734c7d2338d2b2268becb25e996e68aaa64880714173fba6439b652
  Stored in directory: /root/.cache/pip/wheels/09/b6/7e/08b4ee715a1239453e89a59081f0ac369a9036f232e013ecd8

[2020-02-13 14:35:06,538] INFO -   Building wheel for Mako (setup.py): started

|[2020-02-13 14:35:06,871] INFO -   Building wheel for Mako (setup.py): finished with status 'done'

[2020-02-13 14:35:06,872] INFO -   Created wheel for Mako: filename=Mako-1.1.1-py3-none-any.whl size=75409 sha256=390d1d9ec24fb4b66662491e70dbef3c9593c163a0cfdf90f6355611539b5113

[2020-02-13 14:35:06,872] INFO -   Stored in directory: /root/.cache/pip/wheels/11/fe/fa/3693b62cf5ec2b2784b6496734f0ee3e2321eb66d66607e5f9

[2020-02-13 14:35:06,874] INFO - Successfully built cerberus prometheus-client alembic sqlalchemy python-json-logger tabulate Mako

\[2020-02-13 14:35:07,387] INFO - Installing collected packages: MarkupSafe, Jinja2, click, itsdangerous, Werkzeug, flask, cerberus, pyparsing, packaging, humanfriendly, python-dateutil, pytz, pandas, websocket-client, docker, configparser, jmespath, docutils, botocore, s3transfer, boto3, prometheus-client, sqlalchemy, Mako, python-editor, alembic, protobuf, grpcio, ruamel.yaml.clib, ruamel.yaml, python-json-logger, tabulate, bentoml, torch, pillow, torchvision, joblib, scikit-learn, imageio

|[2020-02-13 14:35:45,338] INFO - Successfully installed Jinja2-2.11.1 Mako-1.1.1 MarkupSafe-1.1.1 Werkzeug-1.0.0 alembic-1.4.0 bentoml-0.6.2 boto3-1.11.17 botocore-1.14.17 cerberus-1.3.2 click-7.0 configparser-4.0.2 docker-4.2.0 docutils-0.15.2 flask-1.1.1 grpcio-1.27.1 humanfriendly-6.1 imageio-2.6.1 itsdangerous-1.1.0 jmespath-0.9.4 joblib-0.14.1 packaging-20.1 pandas-1.0.1 pillow-7.0.0 prometheus-client-0.7.1 protobuf-3.11.3 pyparsing-2.4.6 python-dateutil-2.8.0 python-editor-1.0.4 python-json-logger-0.1.11 pytz-2019.3 ruamel.yaml-0.16.10 ruamel.yaml.clib-0.2.0 s3transfer-0.3.3 scikit-learn-0.22.1 sqlalchemy-1.3.13 tabulate-0.8.6 torch-1.4.0 torchvision-0.5.0 websocket-client-0.57.0

/[2020-02-13 14:36:17,983] INFO -  ---> 7bcaa2552bdd

[2020-02-13 14:36:17,983] INFO - Step 9/11 : RUN if [ -f /bento/bentoml_init.sh ]; then /bin/bash -c /bento/bentoml_init.sh; fi
[2020-02-13 14:36:17,985] INFO - 

\[2020-02-13 14:36:18,136] INFO -  ---> Running in 56b0bfa6b9c4

/[2020-02-13 14:36:19,559] INFO -  ---> 9939648103b2

[2020-02-13 14:36:19,560] INFO - Step 10/11 : RUN if [ -f /opt/program/setup.sh ]; then /bin/bash -c /opt/program/setup.sh; fi
[2020-02-13 14:36:19,560] INFO - 

|[2020-02-13 14:36:19,737] INFO -  ---> Running in a271178c15c3

/[2020-02-13 14:36:21,218] INFO -  ---> 6a8c3da0635f

[2020-02-13 14:36:21,218] INFO - Step 11/11 : ENV PATH="/opt/program:${PATH}"
[2020-02-13 14:36:21,219] INFO - 

\[2020-02-13 14:36:21,404] INFO -  ---> Running in bd09032aff38

/[2020-02-13 14:36:21,658] INFO -  ---> da1cac6c729f

[2020-02-13 14:36:21,685] INFO - Successfully built da1cac6c729f

[2020-02-13 14:36:21,690] INFO - Successfully tagged 192023623294.dkr.ecr.us-west-2.amazonaws.com/pytorchfashionclassifier-sagemaker:20200213142355_A51D6E

\[2020-02-13 14:44:24,873] INFO - ApplyDeployment (pytorch-fashion, namespace bobo) succeeded
Successfully created AWS Sagemaker deployment pytorch-fashion
{
  "namespace": "bobo",
  "name": "pytorch-fashion",
  "spec": {
    "bentoName": "PyTorchFashionClassifier",
    "bentoVersion": "20200213142355_A51D6E",
    "operator": "AWS_SAGEMAKER",
    "sagemakerOperatorConfig": {
      "region": "us-west-2",
      "instanceType": "ml.m4.xlarge",
      "instanceCount": 1,
      "apiName": "predict"
    }
  },
  "state": {
    "state": "RUNNING",
    "infoJson": {
      "EndpointName": "bobo-pytorch-fashion",
      "EndpointArn": "arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-pytorch-fashion",
      "EndpointConfigName": "bobo-pytorch-fash-PyTorchFashionClassi-20200213142355-A51D6E",
      "ProductionVariants": [
        {
          "VariantName": "bobo-pytorch-fash-PyTorchFashionClassi-20200213142355-A51D6E",
          "DeployedImages": [
            {
              "SpecifiedImage": "192023623294.dkr.ecr.us-west-2.amazonaws.com/pytorchfashionclassifier-sagemaker:20200213142355_A51D6E",
              "ResolvedImage": "192023623294.dkr.ecr.us-west-2.amazonaws.com/[email protected]:6b69da9461ea2a0c8304b53a77629cac2e67ba98c9a8ab98c9f7623183a27d34",
              "ResolutionTime": "2020-02-13 14:44:28.351000-08:00"
            }
          ],
          "CurrentWeight": 1.0,
          "DesiredWeight": 1.0,
          "CurrentInstanceCount": 1,
          "DesiredInstanceCount": 1
        }
      ],
      "EndpointStatus": "InService",
      "CreationTime": "2020-02-13 14:44:24.788000-08:00",
      "LastModifiedTime": "2020-02-13 14:53:35.549000-08:00",
      "ResponseMetadata": {
        "RequestId": "4f042573-3809-43cb-bb58-9c7008fe6640",
        "HTTPStatusCode": 200,
        "HTTPHeaders": {
          "x-amzn-requestid": "4f042573-3809-43cb-bb58-9c7008fe6640",
          "content-type": "application/x-amz-json-1.1",
          "content-length": "845",
          "date": "Thu, 13 Feb 2020 22:53:40 GMT"
        },
        "RetryAttempts": 0
      }
    },
    "timestamp": "2020-02-13T22:53:40.538570Z"
  },
  "createdAt": "2020-02-13T22:30:58.546725Z",
  "lastUpdatedAt": "2020-02-13T22:30:58.546756Z"
}

bentoml sagemaker list lists all Sagemaker deployments

In [31]:
!bentoml sagemaker list
NAME             NAMESPACE    PLATFORM       BENTO_SERVICE                                   STATUS    AGE
pytorch-fashion  bobo         aws-sagemaker  PyTorchFashionClassifier:20200213142355_A51D6E  running   22 minutes and 57.12 seconds
In [34]:
!bentoml sagemaker get pytorch-fashion
{
  "namespace": "bobo",
  "name": "pytorch-fashion",
  "spec": {
    "bentoName": "PyTorchFashionClassifier",
    "bentoVersion": "20200213142355_A51D6E",
    "operator": "AWS_SAGEMAKER",
    "sagemakerOperatorConfig": {
      "region": "us-west-2",
      "instanceType": "ml.m4.xlarge",
      "instanceCount": 1,
      "apiName": "predict"
    }
  },
  "state": {
    "state": "RUNNING",
    "infoJson": {
      "EndpointName": "bobo-pytorch-fashion",
      "EndpointArn": "arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-pytorch-fashion",
      "EndpointConfigName": "bobo-pytorch-fash-PyTorchFashionClassi-20200213142355-A51D6E",
      "ProductionVariants": [
        {
          "VariantName": "bobo-pytorch-fash-PyTorchFashionClassi-20200213142355-A51D6E",
          "DeployedImages": [
            {
              "SpecifiedImage": "192023623294.dkr.ecr.us-west-2.amazonaws.com/pytorchfashionclassifier-sagemaker:20200213142355_A51D6E",
              "ResolvedImage": "192023623294.dkr.ecr.us-west-2.amazonaws.com/[email protected]:6b69da9461ea2a0c8304b53a77629cac2e67ba98c9a8ab98c9f7623183a27d34",
              "ResolutionTime": "2020-02-13 14:44:28.351000-08:00"
            }
          ],
          "CurrentWeight": 1.0,
          "DesiredWeight": 1.0,
          "CurrentInstanceCount": 1,
          "DesiredInstanceCount": 1
        }
      ],
      "EndpointStatus": "InService",
      "CreationTime": "2020-02-13 14:44:24.788000-08:00",
      "LastModifiedTime": "2020-02-13 14:53:35.549000-08:00",
      "ResponseMetadata": {
        "RequestId": "ada1ef58-47b4-4b65-a1ac-cfc02c0eb969",
        "HTTPStatusCode": 200,
        "HTTPHeaders": {
          "x-amzn-requestid": "ada1ef58-47b4-4b65-a1ac-cfc02c0eb969",
          "content-type": "application/x-amz-json-1.1",
          "content-length": "845",
          "date": "Thu, 13 Feb 2020 22:54:43 GMT"
        },
        "RetryAttempts": 0
      }
    },
    "timestamp": "2020-02-13T22:54:43.436462Z"
  },
  "createdAt": "2020-02-13T22:30:58.546725Z",
  "lastUpdatedAt": "2020-02-13T22:30:58.546756Z"
}

Test and validate Sagemaker deployment with aws sagemaker-runtime invoke-endpoint command

In [35]:
!aws sagemaker-runtime invoke-endpoint --endpoint-name bobo-pytorch-fashion \
--body fileb:///Users/bozhaoyu/src/bento_gallery/pytorch/fashion-mnist/sample_image.png \
--content-type image/png output.json && cat output.json
{
    "ContentType": "application/json",
    "InvokedProductionVariant": "bobo-pytorch-fash-PyTorchFashionClassi-20200213142355-A51D6E"
}
"Bag"

Clean up Sagemaker deployment with bentoml sagemaker delete

In [36]:
!bentoml sagemaker delete pytorch-fashion
Successfully deleted AWS Sagemaker deployment "pytorch-fashion"
In [ ]: