1. gallery
  2. pytorch
  3. cifar10-image-classification

BentoML Example: PyTorch Image Classifier

BentoML is an open source platform for machine learning model serving and deployment. In this project we will use BentoML to package the image classifier model, and build a containerized REST API model server.

This notebook was built based on the official pytorch tutorial https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html

Impression

In [ ]:
%reload_ext autoreload
%autoreload 2
%matplotlib inline
In [ ]:
!pip install bentoml
!pip install torch torchvision pillow
In [1]:
import torch
import torchvision
import torchvision.transforms as transforms

Training an image classifier

We will do the following steps in order:

  • Load and normalizing the CIFAR10 training and test datasets using torchvision
  • Define a Convolutional Neural Network
  • Define a loss function
  • Train the network on the training data
  • Test the network on the test data
  • Model Serving with BentoML

1 Loading and Normalizing CIFAR10

The output of torchvision datasets are PILImage images of range [0, 1]. We transform them to Tensors of normalized range [-1, 1].

In [2]:
transform = transforms.Compose(
    [transforms.ToTensor(),
     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

trainset = torchvision.datasets.CIFAR10(root='./data', train=True,
                                        download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,
                                          shuffle=True, num_workers=2)

testset = torchvision.datasets.CIFAR10(root='./data', train=False,
                                       download=True, transform=transform)
testloader = torch.utils.data.DataLoader(testset, batch_size=4,
                                         shuffle=False, num_workers=2)

classes = ('plane', 'car', 'bird', 'cat',
           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')

Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to ./data/cifar-10-python.tar.gz
Files already downloaded and verified

Let us show some of the training images, for fun.

In [3]:
import matplotlib.pyplot as plt
import numpy as np

# functions to show an image


def imshow(img):
    img = img / 2 + 0.5     # unnormalize
    npimg = img.numpy()
    plt.imshow(np.transpose(npimg, (1, 2, 0)))
    plt.show()


# get some random training images
dataiter = iter(trainloader)
images, labels = dataiter.next()

# show images
imshow(torchvision.utils.make_grid(images))
# print labels
print(' '.join('%5s' % classes[labels[j]] for j in range(4)))

  dog truck   cat plane

2 Define a Convolutional Neural Network

Copy the neural network from the Neural Networks section before and modify it to take 3-channel images (instead of 1-channel images as it was defined).

In [4]:
import torch.nn as nn
import torch.nn.functional as F


class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 5 * 5, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = x.view(-1, 16 * 5 * 5)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x
    
net = Net()

3 Define a Loss function and optimize

Let’s use a Classification Cross-Entropy loss and SGD with momentum.

In [5]:
import torch.optim as optim

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)

4 Train the network

This is when things start to get interesting. We simply have to loop over our data iterator, and feed the inputs to the network and optimize.

In [6]:
for epoch in range(1):  # loop over the dataset multiple times

    running_loss = 0.0
    for i, data in enumerate(trainloader, 0):
        # get the inputs; data is a list of [inputs, labels]
        inputs, labels = data

        # zero the parameter gradients
        optimizer.zero_grad()

        # forward + backward + optimize
        outputs = net(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

        # print statistics
        running_loss += loss.item()
        if i % 2000 == 1999:    # print every 2000 mini-batches
            print('[%d, %5d] loss: %.3f' %
                  (epoch + 1, i + 1, running_loss / 2000))
            running_loss = 0.0

print('Finished Training')

[1,  2000] loss: 2.145
[1,  4000] loss: 1.810
[1,  6000] loss: 1.657
[1,  8000] loss: 1.583
[1, 10000] loss: 1.525
[1, 12000] loss: 1.506
Finished Training

5 Test the network on the test data

We have trained the network for 2 passes over the training dataset. But we need to check if the network has learnt anything at all.

We will check this by predicting the class label that the neural network outputs, and checking it against the ground-truth. If the prediction is correct, we add the sample to the list of correct predictions.

Okay, first step. Let us display an image from the test set to get familiar.

In [7]:
dataiter = iter(testloader)
images, labels = dataiter.next()

# print images
imshow(torchvision.utils.make_grid(images))
print('GroundTruth: ', ' '.join('%5s' % classes[labels[j]] for j in range(4)))

GroundTruth:    cat  ship  ship plane

Let us look at how the network performs on the whole dataset.

In [8]:
correct = 0
total = 0
with torch.no_grad():
    for data in testloader:
        images, labels = data
        outputs = net(images)
        _, predicted = torch.max(outputs.data, 1)
        total += labels.size(0)
        correct += (predicted == labels).sum().item()

print('Accuracy of the network on the 10000 test images: %d %%' % (
    100 * correct / total))

Accuracy of the network on the 10000 test images: 47 %

6 Model Serving with BentoML

First, define ML service with BentoML

In [9]:
%%writefile pytorch_image_classifier.py

from PIL import Image

import torch
from torch.autograd import Variable
from torchvision import transforms

import bentoml
from bentoml.artifact import PytorchModelArtifact
from bentoml.adapters import FileInput


classes = ('plane', 'car', 'bird', 'cat',
           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')

@bentoml.env(pip_dependencies=['torch', 'numpy', 'torchvision', 'scikit-learn'])
@bentoml.artifacts([PytorchModelArtifact('net')])
class PytorchImageClassifier(bentoml.BentoService):
    
    @bentoml.utils.cached_property
    def transform(self):
        return transforms.Compose([
            transforms.ToTensor(),
            transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
        ])
    
    @bentoml.api(input=FileInput())
    def predict(self, file_streams):
        input_datas = []
        for fs in file_streams:
            img = Image.open(fs).resize((32, 32))
            input_datas.append(self.transform(img))

        outputs = self.artifacts.net(Variable(torch.stack(input_datas)))
        _, output_classes = outputs.max(dim=1)

        return [classes[output_class] for output_class in output_classes]

Overwriting pytorch_image_classifier.py

Now you can instantiate a BentoService defined above with the trained model, and save the entire BentoService to a file archive:

In [10]:
# 1) import the custom BentoService defined above
from pytorch_image_classifier import PytorchImageClassifier

# 2) `pack` it with required artifacts
bento_svc = PytorchImageClassifier()
bento_svc.pack('net', net)

# 3) save your BentoSerivce to file archive
saved_path = bento_svc.save()
print(saved_path)

[2020-08-03 17:55:33,425] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.
[2020-08-03 17:55:33,573] WARNING - BentoML by default does not include spacy and torchvision package when using PytorchModelArtifact. To make sure BentoML bundle those packages if they are required for your model, either import those packages in BentoService definition file or manually add them via `@env(pip_dependencies=['torchvision'])` when defining a BentoService
[2020-08-03 17:55:44,087] INFO - Detect BentoML installed in development model, copying local BentoML module file to target saved bundle path
running sdist
running egg_info
writing BentoML.egg-info/PKG-INFO
writing dependency_links to BentoML.egg-info/dependency_links.txt
writing entry points to BentoML.egg-info/entry_points.txt
writing requirements to BentoML.egg-info/requires.txt
writing top-level names to BentoML.egg-info/top_level.txt
reading manifest file 'BentoML.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'

warning: no previously-included files matching '*~' found anywhere in distribution
warning: no previously-included files matching '*.pyo' found anywhere in distribution
warning: no previously-included files matching '.git' found anywhere in distribution
warning: no previously-included files matching '.ipynb_checkpoints' found anywhere in distribution
warning: no previously-included files matching '__pycache__' found anywhere in distribution
warning: no directories found matching 'bentoml/server/static'
warning: no directories found matching 'bentoml/yatai/web/dist'
no previously-included directories found matching 'e2e_tests'
no previously-included directories found matching 'tests'
no previously-included directories found matching 'benchmark'

writing manifest file 'BentoML.egg-info/SOURCES.txt'
running check
creating BentoML-0.8.3+47.g5daa71b
creating BentoML-0.8.3+47.g5daa71b/BentoML.egg-info
creating BentoML-0.8.3+47.g5daa71b/bentoml
creating BentoML-0.8.3+47.g5daa71b/bentoml/adapters
creating BentoML-0.8.3+47.g5daa71b/bentoml/artifact
creating BentoML-0.8.3+47.g5daa71b/bentoml/cli
creating BentoML-0.8.3+47.g5daa71b/bentoml/clipper
creating BentoML-0.8.3+47.g5daa71b/bentoml/configuration
creating BentoML-0.8.3+47.g5daa71b/bentoml/configuration/__pycache__
creating BentoML-0.8.3+47.g5daa71b/bentoml/handlers
creating BentoML-0.8.3+47.g5daa71b/bentoml/marshal
creating BentoML-0.8.3+47.g5daa71b/bentoml/saved_bundle
creating BentoML-0.8.3+47.g5daa71b/bentoml/server
creating BentoML-0.8.3+47.g5daa71b/bentoml/utils
creating BentoML-0.8.3+47.g5daa71b/bentoml/yatai
creating BentoML-0.8.3+47.g5daa71b/bentoml/yatai/client
creating BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment
creating BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/aws_lambda
creating BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/azure_functions
creating BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/sagemaker
creating BentoML-0.8.3+47.g5daa71b/bentoml/yatai/migrations
creating BentoML-0.8.3+47.g5daa71b/bentoml/yatai/migrations/__pycache__
creating BentoML-0.8.3+47.g5daa71b/bentoml/yatai/migrations/versions
creating BentoML-0.8.3+47.g5daa71b/bentoml/yatai/migrations/versions/__pycache__
creating BentoML-0.8.3+47.g5daa71b/bentoml/yatai/proto
creating BentoML-0.8.3+47.g5daa71b/bentoml/yatai/repository
creating BentoML-0.8.3+47.g5daa71b/bentoml/yatai/validator
copying files to BentoML-0.8.3+47.g5daa71b...
copying LICENSE -> BentoML-0.8.3+47.g5daa71b
copying MANIFEST.in -> BentoML-0.8.3+47.g5daa71b
copying README.md -> BentoML-0.8.3+47.g5daa71b
copying pyproject.toml -> BentoML-0.8.3+47.g5daa71b
copying setup.cfg -> BentoML-0.8.3+47.g5daa71b
copying setup.py -> BentoML-0.8.3+47.g5daa71b
copying versioneer.py -> BentoML-0.8.3+47.g5daa71b
copying BentoML.egg-info/PKG-INFO -> BentoML-0.8.3+47.g5daa71b/BentoML.egg-info
copying BentoML.egg-info/SOURCES.txt -> BentoML-0.8.3+47.g5daa71b/BentoML.egg-info
copying BentoML.egg-info/dependency_links.txt -> BentoML-0.8.3+47.g5daa71b/BentoML.egg-info
copying BentoML.egg-info/entry_points.txt -> BentoML-0.8.3+47.g5daa71b/BentoML.egg-info
copying BentoML.egg-info/requires.txt -> BentoML-0.8.3+47.g5daa71b/BentoML.egg-info
copying BentoML.egg-info/top_level.txt -> BentoML-0.8.3+47.g5daa71b/BentoML.egg-info
copying bentoml/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml
copying bentoml/_version.py -> BentoML-0.8.3+47.g5daa71b/bentoml
copying bentoml/exceptions.py -> BentoML-0.8.3+47.g5daa71b/bentoml
copying bentoml/service.py -> BentoML-0.8.3+47.g5daa71b/bentoml
copying bentoml/service_env.py -> BentoML-0.8.3+47.g5daa71b/bentoml
copying bentoml/adapters/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml/adapters
copying bentoml/adapters/base_input.py -> BentoML-0.8.3+47.g5daa71b/bentoml/adapters
copying bentoml/adapters/base_output.py -> BentoML-0.8.3+47.g5daa71b/bentoml/adapters
copying bentoml/adapters/clipper_input.py -> BentoML-0.8.3+47.g5daa71b/bentoml/adapters
copying bentoml/adapters/dataframe_input.py -> BentoML-0.8.3+47.g5daa71b/bentoml/adapters
copying bentoml/adapters/dataframe_output.py -> BentoML-0.8.3+47.g5daa71b/bentoml/adapters
copying bentoml/adapters/default_output.py -> BentoML-0.8.3+47.g5daa71b/bentoml/adapters
copying bentoml/adapters/fastai_image_input.py -> BentoML-0.8.3+47.g5daa71b/bentoml/adapters
copying bentoml/adapters/file_input.py -> BentoML-0.8.3+47.g5daa71b/bentoml/adapters
copying bentoml/adapters/image_input.py -> BentoML-0.8.3+47.g5daa71b/bentoml/adapters
copying bentoml/adapters/json_input.py -> BentoML-0.8.3+47.g5daa71b/bentoml/adapters
copying bentoml/adapters/json_output.py -> BentoML-0.8.3+47.g5daa71b/bentoml/adapters
copying bentoml/adapters/legacy_image_input.py -> BentoML-0.8.3+47.g5daa71b/bentoml/adapters
copying bentoml/adapters/legacy_json_input.py -> BentoML-0.8.3+47.g5daa71b/bentoml/adapters
copying bentoml/adapters/multi_image_input.py -> BentoML-0.8.3+47.g5daa71b/bentoml/adapters
copying bentoml/adapters/pytorch_tensor_input.py -> BentoML-0.8.3+47.g5daa71b/bentoml/adapters
copying bentoml/adapters/tensorflow_tensor_input.py -> BentoML-0.8.3+47.g5daa71b/bentoml/adapters
copying bentoml/adapters/tensorflow_tensor_output.py -> BentoML-0.8.3+47.g5daa71b/bentoml/adapters
copying bentoml/adapters/utils.py -> BentoML-0.8.3+47.g5daa71b/bentoml/adapters
copying bentoml/artifact/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml/artifact
copying bentoml/artifact/artifact.py -> BentoML-0.8.3+47.g5daa71b/bentoml/artifact
copying bentoml/artifact/fastai2_model_artifact.py -> BentoML-0.8.3+47.g5daa71b/bentoml/artifact
copying bentoml/artifact/fastai_model_artifact.py -> BentoML-0.8.3+47.g5daa71b/bentoml/artifact
copying bentoml/artifact/fasttext_model_artifact.py -> BentoML-0.8.3+47.g5daa71b/bentoml/artifact
copying bentoml/artifact/h2o_model_artifact.py -> BentoML-0.8.3+47.g5daa71b/bentoml/artifact
copying bentoml/artifact/json_artifact.py -> BentoML-0.8.3+47.g5daa71b/bentoml/artifact
copying bentoml/artifact/keras_model_artifact.py -> BentoML-0.8.3+47.g5daa71b/bentoml/artifact
copying bentoml/artifact/lightgbm_model_artifact.py -> BentoML-0.8.3+47.g5daa71b/bentoml/artifact
copying bentoml/artifact/onnx_model_artifact.py -> BentoML-0.8.3+47.g5daa71b/bentoml/artifact
copying bentoml/artifact/pickle_artifact.py -> BentoML-0.8.3+47.g5daa71b/bentoml/artifact
copying bentoml/artifact/pytorch_model_artifact.py -> BentoML-0.8.3+47.g5daa71b/bentoml/artifact
copying bentoml/artifact/sklearn_model_artifact.py -> BentoML-0.8.3+47.g5daa71b/bentoml/artifact
copying bentoml/artifact/spacy_model_artifact.py -> BentoML-0.8.3+47.g5daa71b/bentoml/artifact
copying bentoml/artifact/text_file_artifact.py -> BentoML-0.8.3+47.g5daa71b/bentoml/artifact
copying bentoml/artifact/tf_savedmodel_artifact.py -> BentoML-0.8.3+47.g5daa71b/bentoml/artifact
copying bentoml/artifact/xgboost_model_artifact.py -> BentoML-0.8.3+47.g5daa71b/bentoml/artifact
copying bentoml/cli/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml/cli
copying bentoml/cli/aws_lambda.py -> BentoML-0.8.3+47.g5daa71b/bentoml/cli
copying bentoml/cli/aws_sagemaker.py -> BentoML-0.8.3+47.g5daa71b/bentoml/cli
copying bentoml/cli/azure_functions.py -> BentoML-0.8.3+47.g5daa71b/bentoml/cli
copying bentoml/cli/bento_management.py -> BentoML-0.8.3+47.g5daa71b/bentoml/cli
copying bentoml/cli/bento_service.py -> BentoML-0.8.3+47.g5daa71b/bentoml/cli
copying bentoml/cli/click_utils.py -> BentoML-0.8.3+47.g5daa71b/bentoml/cli
copying bentoml/cli/config.py -> BentoML-0.8.3+47.g5daa71b/bentoml/cli
copying bentoml/cli/deployment.py -> BentoML-0.8.3+47.g5daa71b/bentoml/cli
copying bentoml/cli/utils.py -> BentoML-0.8.3+47.g5daa71b/bentoml/cli
copying bentoml/cli/yatai_service.py -> BentoML-0.8.3+47.g5daa71b/bentoml/cli
copying bentoml/clipper/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml/clipper
copying bentoml/configuration/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml/configuration
copying bentoml/configuration/configparser.py -> BentoML-0.8.3+47.g5daa71b/bentoml/configuration
copying bentoml/configuration/default_bentoml.cfg -> BentoML-0.8.3+47.g5daa71b/bentoml/configuration
copying bentoml/configuration/__pycache__/__init__.cpython-36.pyc -> BentoML-0.8.3+47.g5daa71b/bentoml/configuration/__pycache__
copying bentoml/configuration/__pycache__/__init__.cpython-37.pyc -> BentoML-0.8.3+47.g5daa71b/bentoml/configuration/__pycache__
copying bentoml/configuration/__pycache__/__init__.cpython-38.pyc -> BentoML-0.8.3+47.g5daa71b/bentoml/configuration/__pycache__
copying bentoml/configuration/__pycache__/configparser.cpython-36.pyc -> BentoML-0.8.3+47.g5daa71b/bentoml/configuration/__pycache__
copying bentoml/configuration/__pycache__/configparser.cpython-37.pyc -> BentoML-0.8.3+47.g5daa71b/bentoml/configuration/__pycache__
copying bentoml/configuration/__pycache__/configparser.cpython-38.pyc -> BentoML-0.8.3+47.g5daa71b/bentoml/configuration/__pycache__
copying bentoml/handlers/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml/handlers
copying bentoml/marshal/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml/marshal
copying bentoml/marshal/dispatcher.py -> BentoML-0.8.3+47.g5daa71b/bentoml/marshal
copying bentoml/marshal/marshal.py -> BentoML-0.8.3+47.g5daa71b/bentoml/marshal
copying bentoml/marshal/utils.py -> BentoML-0.8.3+47.g5daa71b/bentoml/marshal
copying bentoml/saved_bundle/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml/saved_bundle
copying bentoml/saved_bundle/bentoml-init.sh -> BentoML-0.8.3+47.g5daa71b/bentoml/saved_bundle
copying bentoml/saved_bundle/bundler.py -> BentoML-0.8.3+47.g5daa71b/bentoml/saved_bundle
copying bentoml/saved_bundle/config.py -> BentoML-0.8.3+47.g5daa71b/bentoml/saved_bundle
copying bentoml/saved_bundle/docker-entrypoint.sh -> BentoML-0.8.3+47.g5daa71b/bentoml/saved_bundle
copying bentoml/saved_bundle/loader.py -> BentoML-0.8.3+47.g5daa71b/bentoml/saved_bundle
copying bentoml/saved_bundle/pip_pkg.py -> BentoML-0.8.3+47.g5daa71b/bentoml/saved_bundle
copying bentoml/saved_bundle/py_module_utils.py -> BentoML-0.8.3+47.g5daa71b/bentoml/saved_bundle
copying bentoml/saved_bundle/templates.py -> BentoML-0.8.3+47.g5daa71b/bentoml/saved_bundle
copying bentoml/server/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml/server
copying bentoml/server/api_server.py -> BentoML-0.8.3+47.g5daa71b/bentoml/server
copying bentoml/server/gunicorn_config.py -> BentoML-0.8.3+47.g5daa71b/bentoml/server
copying bentoml/server/gunicorn_server.py -> BentoML-0.8.3+47.g5daa71b/bentoml/server
copying bentoml/server/instruments.py -> BentoML-0.8.3+47.g5daa71b/bentoml/server
copying bentoml/server/marshal_server.py -> BentoML-0.8.3+47.g5daa71b/bentoml/server
copying bentoml/server/open_api.py -> BentoML-0.8.3+47.g5daa71b/bentoml/server
copying bentoml/server/trace.py -> BentoML-0.8.3+47.g5daa71b/bentoml/server
copying bentoml/server/utils.py -> BentoML-0.8.3+47.g5daa71b/bentoml/server
copying bentoml/utils/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml/utils
copying bentoml/utils/alg.py -> BentoML-0.8.3+47.g5daa71b/bentoml/utils
copying bentoml/utils/benchmark.py -> BentoML-0.8.3+47.g5daa71b/bentoml/utils
copying bentoml/utils/cloudpickle.py -> BentoML-0.8.3+47.g5daa71b/bentoml/utils
copying bentoml/utils/dataframe_util.py -> BentoML-0.8.3+47.g5daa71b/bentoml/utils
copying bentoml/utils/flask_ngrok.py -> BentoML-0.8.3+47.g5daa71b/bentoml/utils
copying bentoml/utils/hybridmethod.py -> BentoML-0.8.3+47.g5daa71b/bentoml/utils
copying bentoml/utils/lazy_loader.py -> BentoML-0.8.3+47.g5daa71b/bentoml/utils
copying bentoml/utils/log.py -> BentoML-0.8.3+47.g5daa71b/bentoml/utils
copying bentoml/utils/s3.py -> BentoML-0.8.3+47.g5daa71b/bentoml/utils
copying bentoml/utils/tempdir.py -> BentoML-0.8.3+47.g5daa71b/bentoml/utils
copying bentoml/utils/usage_stats.py -> BentoML-0.8.3+47.g5daa71b/bentoml/utils
copying bentoml/yatai/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai
copying bentoml/yatai/alembic.ini -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai
copying bentoml/yatai/db.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai
copying bentoml/yatai/deployment_utils.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai
copying bentoml/yatai/status.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai
copying bentoml/yatai/utils.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai
copying bentoml/yatai/yatai_service.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai
copying bentoml/yatai/yatai_service_impl.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai
copying bentoml/yatai/client/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/client
copying bentoml/yatai/client/bento_repository_api.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/client
copying bentoml/yatai/client/deployment_api.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/client
copying bentoml/yatai/deployment/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment
copying bentoml/yatai/deployment/operator.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment
copying bentoml/yatai/deployment/store.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment
copying bentoml/yatai/deployment/utils.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment
copying bentoml/yatai/deployment/aws_lambda/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/aws_lambda
copying bentoml/yatai/deployment/aws_lambda/download_extra_resources.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/aws_lambda
copying bentoml/yatai/deployment/aws_lambda/lambda_app.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/aws_lambda
copying bentoml/yatai/deployment/aws_lambda/operator.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/aws_lambda
copying bentoml/yatai/deployment/aws_lambda/utils.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/aws_lambda
copying bentoml/yatai/deployment/azure_functions/Dockerfile -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/azure_functions
copying bentoml/yatai/deployment/azure_functions/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/azure_functions
copying bentoml/yatai/deployment/azure_functions/app_init.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/azure_functions
copying bentoml/yatai/deployment/azure_functions/constants.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/azure_functions
copying bentoml/yatai/deployment/azure_functions/host.json -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/azure_functions
copying bentoml/yatai/deployment/azure_functions/local.settings.json -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/azure_functions
copying bentoml/yatai/deployment/azure_functions/operator.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/azure_functions
copying bentoml/yatai/deployment/azure_functions/templates.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/azure_functions
copying bentoml/yatai/deployment/sagemaker/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/sagemaker
copying bentoml/yatai/deployment/sagemaker/model_server.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/sagemaker
copying bentoml/yatai/deployment/sagemaker/nginx.conf -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/sagemaker
copying bentoml/yatai/deployment/sagemaker/operator.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/sagemaker
copying bentoml/yatai/deployment/sagemaker/serve -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/sagemaker
copying bentoml/yatai/deployment/sagemaker/wsgi.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/deployment/sagemaker
copying bentoml/yatai/migrations/README -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/migrations
copying bentoml/yatai/migrations/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/migrations
copying bentoml/yatai/migrations/env.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/migrations
copying bentoml/yatai/migrations/script.py.mako -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/migrations
copying bentoml/yatai/migrations/__pycache__/env.cpython-36.pyc -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/migrations/__pycache__
copying bentoml/yatai/migrations/versions/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/migrations/versions
copying bentoml/yatai/migrations/versions/a6b00ae45279_add_last_updated_at_for_deployments.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/migrations/versions
copying bentoml/yatai/migrations/versions/__pycache__/a6b00ae45279_add_last_updated_at_for_deployments.cpython-36.pyc -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/migrations/versions/__pycache__
copying bentoml/yatai/proto/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/proto
copying bentoml/yatai/proto/deployment_pb2.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/proto
copying bentoml/yatai/proto/repository_pb2.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/proto
copying bentoml/yatai/proto/status_pb2.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/proto
copying bentoml/yatai/proto/yatai_service_pb2.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/proto
copying bentoml/yatai/proto/yatai_service_pb2_grpc.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/proto
copying bentoml/yatai/repository/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/repository
copying bentoml/yatai/repository/base_repository.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/repository
copying bentoml/yatai/repository/local_repository.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/repository
copying bentoml/yatai/repository/metadata_store.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/repository
copying bentoml/yatai/repository/repository.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/repository
copying bentoml/yatai/repository/s3_repository.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/repository
copying bentoml/yatai/validator/__init__.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/validator
copying bentoml/yatai/validator/deployment_pb_validator.py -> BentoML-0.8.3+47.g5daa71b/bentoml/yatai/validator
Writing BentoML-0.8.3+47.g5daa71b/setup.cfg
UPDATING BentoML-0.8.3+47.g5daa71b/bentoml/_version.py
set BentoML-0.8.3+47.g5daa71b/bentoml/_version.py to '0.8.3+47.g5daa71b'
Creating tar archive
removing 'BentoML-0.8.3+47.g5daa71b' (and everything under it)
[2020-08-03 17:55:45,086] INFO - BentoService bundle 'PytorchImageClassifier:20200803175534_4C7106' saved to: /home/bentoml/bentoml/repository/PytorchImageClassifier/20200803175534_4C7106
/home/bentoml/bentoml/repository/PytorchImageClassifier/20200803175534_4C7106

BentoService with BentoML CLI

In [12]:
!bentoml get PytorchImageClassifier

BENTO_SERVICE                                 CREATED_AT        APIS                   ARTIFACTS
PytorchImageClassifier:20200127115720_71B982  2020-01-27 19:57  predict::ImageHandler  net::PytorchModelArtifact
PytorchImageClassifier:20200127114517_C3118B  2020-01-27 19:45  predict::ImageHandler  net::PytorchModelArtifact
In [11]:
!bentoml get PytorchImageClassifier:latest

[2020-08-03 17:55:51,187] INFO - Getting latest version PytorchImageClassifier:20200803175534_4C7106
{
  "name": "PytorchImageClassifier",
  "version": "20200803175534_4C7106",
  "uri": {
    "type": "LOCAL",
    "uri": "/home/bentoml/bentoml/repository/PytorchImageClassifier/20200803175534_4C7106"
  },
  "bentoServiceMetadata": {
    "name": "PytorchImageClassifier",
    "version": "20200803175534_4C7106",
    "createdAt": "2020-08-03T09:55:45.058786Z",
    "env": {
      "condaEnv": "name: bentoml-PytorchImageClassifier\nchannels:\n- defaults\ndependencies:\n- python=3.6.10\n- pip\n",
      "pipDependencies": "torch\nbentoml==0.8.3\nnumpy\ntorchvision\nscikit-learn",
      "pythonVersion": "3.6.10",
      "dockerBaseImage": "bentoml/model-server:0.8.3"
    },
    "artifacts": [
      {
        "name": "net",
        "artifactType": "PytorchModelArtifact"
      }
    ],
    "apis": [
      {
        "name": "predict",
        "inputType": "FileInput",
        "docs": "BentoService inference API 'predict', input: 'FileInput', output: 'DefaultOutput'",
        "outputConfig": {
          "cors": "*"
        },
        "outputType": "DefaultOutput",
        "mbMaxLatency": 10000,
        "mbMaxBatchSize": 2000
      }
    ]
  }
}
In [12]:
!bentoml info PytorchImageClassifier:latest

[2020-08-03 17:56:08,207] INFO - Getting latest version PytorchImageClassifier:20200803175534_4C7106
[2020-08-03 17:56:08,227] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.
[2020-08-03 17:56:08,242] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.8.3, but loading from BentoML version 0.8.3+47.g5daa71b
{
  "name": "PytorchImageClassifier",
  "version": "20200803175534_4C7106",
  "created_at": "2020-08-03T09:55:44.025755Z",
  "env": {
    "conda_env": "name: bentoml-PytorchImageClassifier\nchannels:\n- defaults\ndependencies:\n- python=3.6.10\n- pip\n",
    "pip_dependencies": "torch\nbentoml==0.8.3\nnumpy\ntorchvision\nscikit-learn",
    "python_version": "3.6.10",
    "docker_base_image": "bentoml/model-server:0.8.3"
  },
  "artifacts": [
    {
      "name": "net",
      "artifact_type": "PytorchModelArtifact"
    }
  ],
  "apis": [
    {
      "name": "predict",
      "input_type": "FileInput",
      "docs": "BentoService inference API 'predict', input: 'FileInput', output: 'DefaultOutput'",
      "output_config": {
        "cors": "*"
      },
      "output_type": "DefaultOutput",
      "mb_max_latency": 10000,
      "mb_max_batch_size": 2000
    }
  ]
}
In [13]:
!bentoml run PytorchImageClassifier:latest predict  --input=bird.jpg

[2020-08-03 17:56:18,947] INFO - Getting latest version PytorchImageClassifier:20200803175534_4C7106
[2020-08-03 17:56:20,021] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.
[2020-08-03 17:56:20,038] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.8.3, but loading from BentoML version 0.8.3+47.g5daa71b
[2020-08-03 17:56:20,832] WARNING - BentoML by default does not include spacy and torchvision package when using PytorchModelArtifact. To make sure BentoML bundle those packages if they are required for your model, either import those packages in BentoService definition file or manually add them via `@env(pip_dependencies=['torchvision'])` when defining a BentoService
deer

Model Serving via REST API

In your termnial, run the following command to start the REST API server:

In [14]:
!bentoml serve PytorchImageClassifier:latest --enable-microbatch

[2020-08-03 17:56:58,854] INFO - Getting latest version PytorchImageClassifier:20200803175534_4C7106
[2020-08-03 17:56:58,855] INFO - Starting BentoML API server in development mode..
[2020-08-03 17:56:59,917] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.
[2020-08-03 17:56:59,940] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.8.3, but loading from BentoML version 0.8.3+47.g5daa71b
[2020-08-03 17:57:00,626] WARNING - BentoML by default does not include spacy and torchvision package when using PytorchModelArtifact. To make sure BentoML bundle those packages if they are required for your model, either import those packages in BentoService definition file or manually add them via `@env(pip_dependencies=['torchvision'])` when defining a BentoService
[2020-08-03 17:57:01,595] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.8.3, but loading from BentoML version 0.8.3+47.g5daa71b
[2020-08-03 17:57:01,596] INFO - Micro batch enabled for API `predict`
[2020-08-03 17:57:01,597] INFO - Your system nofile limit is 10000, which means each instance of microbatch service is able to hold this number of connections at same time. You can increase the number of file descriptors for the server process, or launch more microbatch instances to accept more concurrent connection.
[2020-08-03 17:57:01,606] INFO - Running micro batch service on :5000
 * Serving Flask app "PytorchImageClassifier" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:60135/ (Press CTRL+C to quit)
======== Running on http://0.0.0.0:5000 ========
(Press CTRL+C to quit)
127.0.0.1 - - [03/Aug/2020 17:57:42] "POST /predict HTTP/1.1" 200 -
127.0.0.1 - - [03/Aug/2020 17:57:44] "POST /predict HTTP/1.1" 200 -
127.0.0.1 - - [03/Aug/2020 17:57:45] "POST /predict HTTP/1.1" 200 -
127.0.0.1 - - [03/Aug/2020 17:57:46] "POST /predict HTTP/1.1" 200 -
127.0.0.1 - - [03/Aug/2020 17:57:46] "POST /predict HTTP/1.1" 200 -
^C

Open a terminal and go to the directory of the current notebook, then run the following command to evaluate.

curl -X POST "http://127.0.0.1:5000/predict" -H "Content-Type: image/*" --data-binary "@bird.jpg"

alternatively:

curl -X POST "http://127.0.0.1:5000/predict" -F image=@bird.png

Go visit http://127.0.0.1:5000/ from your browser, click /predict -> Try it out -> Choose File -> Execute to sumbit an image from your computer

Use BentoService as PyPI package

In [15]:
!pip install --quiet {saved_path}
In [16]:
!PytorchImageClassifier run predict --input=bird.jpg

deer

Deploy BentoService as REST API server to the cloud

BentoML support deployment to multiply cloud provider services, such as AWS Lambda, AWS Sagemaker, Google Cloudrun and etc. You can find the full list and guide on the documentation site at https://docs.bentoml.org/en/latest/deployment/index.html

For fastai pet classification, we are going to deploy to AWS Sagemaker

Deploying to Sagemaker with bentoml sagemaker deploy

In [22]:
!bentoml sagemaker deploy first-pytorch -b PytorchImageClassifier:latest --api-name predict

Deploying Sagemaker deployment \[2020-01-27 12:05:30,122] INFO - Step 1/11 : FROM continuumio/miniconda3:4.7.12
[2020-01-27 12:05:30,124] INFO - 

[2020-01-27 12:05:30,125] INFO -  ---> 406f2b43ea59

[2020-01-27 12:05:30,125] INFO - Step 2/11 : EXPOSE 8080
[2020-01-27 12:05:30,125] INFO - 

[2020-01-27 12:05:30,125] INFO -  ---> Using cache

[2020-01-27 12:05:30,125] INFO -  ---> 7c8096d6922f

[2020-01-27 12:05:30,125] INFO - Step 3/11 : RUN set -x      && apt-get update      && apt-get install --no-install-recommends --no-install-suggests -y libpq-dev build-essential     && apt-get install -y nginx      && rm -rf /var/lib/apt/lists/*
[2020-01-27 12:05:30,125] INFO - 

[2020-01-27 12:05:30,126] INFO -  ---> Using cache

[2020-01-27 12:05:30,126] INFO -  ---> 5b00a58f3bce

[2020-01-27 12:05:30,126] INFO - Step 4/11 : RUN conda install pip numpy scipy       && pip install gunicorn gevent
[2020-01-27 12:05:30,126] INFO - 

[2020-01-27 12:05:30,126] INFO -  ---> Using cache

[2020-01-27 12:05:30,127] INFO -  ---> 25aacb592861

[2020-01-27 12:05:30,127] INFO - Step 5/11 : COPY . /opt/program
[2020-01-27 12:05:30,127] INFO - 

[2020-01-27 12:05:30,127] INFO -  ---> Using cache

[2020-01-27 12:05:30,127] INFO -  ---> cad4120a0f7c

[2020-01-27 12:05:30,127] INFO - Step 6/11 : WORKDIR /opt/program
[2020-01-27 12:05:30,127] INFO - 

[2020-01-27 12:05:30,128] INFO -  ---> Using cache

[2020-01-27 12:05:30,128] INFO -  ---> 863e0db936e9

[2020-01-27 12:05:30,128] INFO - Step 7/11 : RUN conda env update -n base -f /opt/program/environment.yml
[2020-01-27 12:05:30,128] INFO - 

[2020-01-27 12:05:30,128] INFO -  ---> Using cache

[2020-01-27 12:05:30,128] INFO -  ---> bf004b1d85ca

[2020-01-27 12:05:30,129] INFO - Step 8/11 : RUN pip install -r /opt/program/requirements.txt
[2020-01-27 12:05:30,129] INFO - 

-[2020-01-27 12:05:30,223] INFO -  ---> Running in 09eba76c8667

\[2020-01-27 12:05:31,838] INFO - Collecting bentoml==0.6.1

-[2020-01-27 12:05:31,941] INFO -   Downloading https://files.pythonhosted.org/packages/95/5b/6dd42537eb3fce0f3f28ccb84435a1d5ea6dd6a5756d120d75a6c47e021d/BentoML-0.6.1-py3-none-any.whl (553kB)

-[2020-01-27 12:05:32,282] INFO - Collecting torch

[2020-01-27 12:05:32,299] INFO -   Downloading https://files.pythonhosted.org/packages/1a/3b/fa92ece1e58a6a48ec598bab327f39d69808133e5b2fb33002ca754e381e/torch-1.4.0-cp37-cp37m-manylinux1_x86_64.whl (753.4MB)

|[2020-01-27 12:06:36,105] INFO - Requirement already satisfied: numpy in /opt/conda/lib/python3.7/site-packages (from -r /opt/program/requirements.txt (line 3)) (1.18.1)

|[2020-01-27 12:06:36,499] INFO - Collecting torchvision

[2020-01-27 12:06:36,528] INFO -   Downloading https://files.pythonhosted.org/packages/1c/32/cb0e4c43cd717da50258887b088471568990b5a749784c465a8a1962e021/torchvision-0.5.0-cp37-cp37m-manylinux1_x86_64.whl (4.0MB)

-[2020-01-27 12:06:37,136] INFO - Collecting scikit-learn

/[2020-01-27 12:06:37,160] INFO -   Downloading https://files.pythonhosted.org/packages/73/db/7d8204ddba84ab5d1e4fd1af8f82bbe39c589488bee71e45c662f4144010/scikit_learn-0.22.1-cp37-cp37m-manylinux1_x86_64.whl (7.0MB)

|[2020-01-27 12:06:37,724] INFO - Collecting imageio

[2020-01-27 12:06:37,750] INFO -   Downloading https://files.pythonhosted.org/packages/1a/de/f7f985018f462ceeffada7f6e609919fbcc934acd9301929cba14bc2c24a/imageio-2.6.1-py3-none-any.whl (3.3MB)

-[2020-01-27 12:06:37,971] INFO - Collecting docker

/[2020-01-27 12:06:37,993] INFO -   Downloading https://files.pythonhosted.org/packages/cc/ca/699d4754a932787ef353a157ada74efd1ceb6d1fc0bfb7989ae1e7b33111/docker-4.1.0-py2.py3-none-any.whl (139kB)

|[2020-01-27 12:06:38,097] INFO - Collecting alembic

[2020-01-27 12:06:38,115] INFO -   Downloading https://files.pythonhosted.org/packages/9d/c9/d4aa3be3511dfd6d86f8f483ce0d9f120258be4aceadc17601843593e2ec/alembic-1.3.3.tar.gz (1.1MB)

/[2020-01-27 12:06:40,123] INFO - Collecting ruamel.yaml>=0.15.0

[2020-01-27 12:06:40,141] INFO -   Downloading https://files.pythonhosted.org/packages/99/a8/25242fc81d0c02460e93e8cc7ec4ae60c2382c00ba0a0e1becc22e76548d/ruamel.yaml-0.16.6-py2.py3-none-any.whl (123kB)

|[2020-01-27 12:06:40,235] INFO - Collecting cerberus

\[2020-01-27 12:06:40,262] INFO -   Downloading https://files.pythonhosted.org/packages/90/a7/71c6ed2d46a81065e68c007ac63378b96fa54c7bb614d653c68232f9c50c/Cerberus-1.3.2.tar.gz (52kB)

/[2020-01-27 12:06:40,510] INFO - Requirement already satisfied: gunicorn in /opt/conda/lib/python3.7/site-packages (from bentoml==0.6.1->-r /opt/program/requirements.txt (line 1)) (20.0.4)

|[2020-01-27 12:06:40,572] INFO - Collecting flask

[2020-01-27 12:06:40,593] INFO -   Downloading https://files.pythonhosted.org/packages/9b/93/628509b8d5dc749656a9641f4caf13540e2cdec85276964ff8f43bbb1d3b/Flask-1.1.1-py2.py3-none-any.whl (94kB)

\[2020-01-27 12:06:40,680] INFO - Collecting python-json-logger

[2020-01-27 12:06:40,701] INFO -   Downloading https://files.pythonhosted.org/packages/80/9d/1c3393a6067716e04e6fcef95104c8426d262b4adaf18d7aa2470eab028d/python-json-logger-0.1.11.tar.gz

|[2020-01-27 12:06:40,973] INFO - Collecting tabulate

[2020-01-27 12:06:40,993] INFO -   Downloading https://files.pythonhosted.org/packages/c4/41/523f6a05e6dc3329a5660f6a81254c6cd87e5cfb5b7482bae3391d86ec3a/tabulate-0.8.6.tar.gz (45kB)

|[2020-01-27 12:06:41,406] INFO - Collecting protobuf>=3.6.0

[2020-01-27 12:06:41,428] INFO -   Downloading https://files.pythonhosted.org/packages/4a/14/f5c294f1e36a031f165128c25feba93b3116f15a74398d0b2747ed75744f/protobuf-3.11.2-cp37-cp37m-manylinux1_x86_64.whl (1.3MB)

\[2020-01-27 12:06:41,575] INFO - Collecting configparser

-[2020-01-27 12:06:41,595] INFO -   Downloading https://files.pythonhosted.org/packages/7a/2a/95ed0501cf5d8709490b1d3a3f9b5cf340da6c433f896bbe9ce08dbe6785/configparser-4.0.2-py2.py3-none-any.whl

|[2020-01-27 12:06:42,292] INFO - Collecting grpcio

\[2020-01-27 12:06:42,321] INFO -   Downloading https://files.pythonhosted.org/packages/bc/b3/0052e38c640d52b710e235b15821cc3c61d0065bf54e70a44550ef127349/grpcio-1.26.0-cp37-cp37m-manylinux2010_x86_64.whl (2.4MB)

\[2020-01-27 12:06:42,814] INFO - Collecting boto3

-[2020-01-27 12:06:42,840] INFO -   Downloading https://files.pythonhosted.org/packages/d5/57/e9675a5a8d0ee586594ff19cb9a601334fbf24fa2fb29052d2a900ee5d23/boto3-1.11.9-py2.py3-none-any.whl (128kB)

/[2020-01-27 12:06:42,969] INFO - Collecting sqlalchemy>=1.3.0

[2020-01-27 12:06:42,991] INFO -   Downloading https://files.pythonhosted.org/packages/af/47/35edeb0f86c0b44934c05d961c893e223ef27e79e1f53b5e6f14820ff553/SQLAlchemy-1.3.13.tar.gz (6.0MB)

|[2020-01-27 12:06:44,367] INFO - Collecting click>=7.0

\[2020-01-27 12:06:44,388] INFO -   Downloading https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl (81kB)

-[2020-01-27 12:06:44,538] INFO - Collecting python-dateutil<2.8.1,>=2.1

[2020-01-27 12:06:44,559] INFO -   Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB)

/[2020-01-27 12:06:44,654] INFO - Collecting packaging

[2020-01-27 12:06:44,673] INFO -   Downloading https://files.pythonhosted.org/packages/98/42/87c585dd3b113c775e65fd6b8d9d0a43abe1819c471d7af702d4e01e9b20/packaging-20.1-py2.py3-none-any.whl

|[2020-01-27 12:06:44,711] INFO - Requirement already satisfied: requests in /opt/conda/lib/python3.7/site-packages (from bentoml==0.6.1->-r /opt/program/requirements.txt (line 1)) (2.22.0)

[2020-01-27 12:06:44,781] INFO - Collecting prometheus-client

\[2020-01-27 12:06:44,798] INFO -   Downloading https://files.pythonhosted.org/packages/b3/23/41a5a24b502d35a4ad50a5bb7202a5e1d9a0364d0c12f56db3dbf7aca76d/prometheus_client-0.7.1.tar.gz

|[2020-01-27 12:06:45,147] INFO - Collecting humanfriendly

[2020-01-27 12:06:45,165] INFO -   Downloading https://files.pythonhosted.org/packages/90/df/88bff450f333114680698dc4aac7506ff7cab164b794461906de31998665/humanfriendly-4.18-py2.py3-none-any.whl (73kB)

/[2020-01-27 12:06:45,450] INFO - Collecting pandas

[2020-01-27 12:06:45,470] INFO -   Downloading https://files.pythonhosted.org/packages/63/e0/a1b39cdcb2c391f087a1538bc8a6d62a82d0439693192aef541d7b123769/pandas-0.25.3-cp37-cp37m-manylinux1_x86_64.whl (10.4MB)

/[2020-01-27 12:06:46,678] INFO - Collecting pillow>=4.1.1

[2020-01-27 12:06:46,702] INFO -   Downloading https://files.pythonhosted.org/packages/f5/79/b2d5695d1a931474fa68b68ec93bdf08ba9acbc4d6b3b628eb6aac81d11c/Pillow-7.0.0-cp37-cp37m-manylinux1_x86_64.whl (2.1MB)

\[2020-01-27 12:06:46,848] INFO - Requirement already satisfied: six in /opt/conda/lib/python3.7/site-packages (from torchvision->-r /opt/program/requirements.txt (line 4)) (1.12.0)

[2020-01-27 12:06:46,924] INFO - Collecting joblib>=0.11

[2020-01-27 12:06:46,946] INFO -   Downloading https://files.pythonhosted.org/packages/28/5c/cf6a2b65a321c4a209efcdf64c2689efae2cb62661f8f6f4bb28547cf1bf/joblib-0.14.1-py2.py3-none-any.whl (294kB)

-[2020-01-27 12:06:47,003] INFO - Requirement already satisfied: scipy>=0.17.0 in /opt/conda/lib/python3.7/site-packages (from scikit-learn->-r /opt/program/requirements.txt (line 5)) (1.3.2)

/[2020-01-27 12:06:47,085] INFO - Collecting websocket-client>=0.32.0

[2020-01-27 12:06:47,116] INFO -   Downloading https://files.pythonhosted.org/packages/4c/5f/f61b420143ed1c8dc69f9eaec5ff1ac36109d52c80de49d66e0c36c3dfdf/websocket_client-0.57.0-py2.py3-none-any.whl (200kB)

\[2020-01-27 12:06:47,334] INFO - Collecting Mako

[2020-01-27 12:06:47,359] INFO -   Downloading https://files.pythonhosted.org/packages/28/03/329b21f00243fc2d3815399413845dbbfb0745cff38a29d3597e97f8be58/Mako-1.1.1.tar.gz (468kB)

-[2020-01-27 12:06:47,837] INFO - Collecting python-editor>=0.3

[2020-01-27 12:06:47,855] INFO -   Downloading https://files.pythonhosted.org/packages/c6/d3/201fc3abe391bbae6606e6f1d598c15d367033332bd54352b12f35513717/python_editor-1.0.4-py3-none-any.whl

/[2020-01-27 12:06:47,914] INFO - Collecting ruamel.yaml.clib>=0.1.2; platform_python_implementation == "CPython" and python_version < "3.8"

[2020-01-27 12:06:47,938] INFO -   Downloading https://files.pythonhosted.org/packages/40/80/da16b691d5e259dd9919a10628e541fca321cb4b078fbb88e1c7c22aa42d/ruamel.yaml.clib-0.2.0-cp37-cp37m-manylinux1_x86_64.whl (547kB)

|[2020-01-27 12:06:47,988] INFO - Requirement already satisfied: setuptools in /opt/conda/lib/python3.7/site-packages (from cerberus->bentoml==0.6.1->-r /opt/program/requirements.txt (line 1)) (41.4.0)

[2020-01-27 12:06:48,056] INFO - Collecting Werkzeug>=0.15

[2020-01-27 12:06:48,076] INFO -   Downloading https://files.pythonhosted.org/packages/c2/e4/a859d2fe516f466642fa5c6054fd9646271f9da26b0cac0d2f37fc858c8f/Werkzeug-0.16.1-py2.py3-none-any.whl (327kB)

\[2020-01-27 12:06:48,165] INFO - Collecting itsdangerous>=0.24

[2020-01-27 12:06:48,183] INFO -   Downloading https://files.pythonhosted.org/packages/76/ae/44b03b253d6fade317f32c24d100b3b35c2239807046a4c953c7b89fa49e/itsdangerous-1.1.0-py2.py3-none-any.whl

/[2020-01-27 12:06:48,298] INFO - Collecting Jinja2>=2.10.1

[2020-01-27 12:06:48,314] INFO -   Downloading https://files.pythonhosted.org/packages/7b/af/b9ed1959cb4bb7332e2b0797476c878fa38d200bfcfe38c6d53517c29bdf/Jinja2-2.11.0-py2.py3-none-any.whl (126kB)

|[2020-01-27 12:06:48,458] INFO - Collecting s3transfer<0.4.0,>=0.3.0

[2020-01-27 12:06:48,492] INFO -   Downloading https://files.pythonhosted.org/packages/c7/48/a8252b6b3cd31774eab312b19d58a6ac55f296240c206617dcd38cd93bf8/s3transfer-0.3.2-py2.py3-none-any.whl (69kB)

/[2020-01-27 12:06:49,226] INFO - Collecting botocore<1.15.0,>=1.14.9

|[2020-01-27 12:06:49,264] INFO -   Downloading https://files.pythonhosted.org/packages/64/4c/b0b0d3b6f84a05f9135051b56d3eb8708012a289c4b82ee21c8c766f47b5/botocore-1.14.9-py2.py3-none-any.whl (5.9MB)

/[2020-01-27 12:06:50,062] INFO - Collecting jmespath<1.0.0,>=0.7.1

|[2020-01-27 12:06:50,079] INFO -   Downloading https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl

\[2020-01-27 12:06:50,192] INFO - Collecting pyparsing>=2.0.2

[2020-01-27 12:06:50,214] INFO -   Downloading https://files.pythonhosted.org/packages/5d/bc/1e58593167fade7b544bfe9502a26dc860940a79ab306e651e7f13be68c2/pyparsing-2.4.6-py2.py3-none-any.whl (67kB)

[2020-01-27 12:06:50,236] INFO - Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.6.1->-r /opt/program/requirements.txt (line 1)) (3.0.4)

[2020-01-27 12:06:50,238] INFO - Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.6.1->-r /opt/program/requirements.txt (line 1)) (2019.11.28)

[2020-01-27 12:06:50,242] INFO - Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.6.1->-r /opt/program/requirements.txt (line 1)) (1.24.2)

[2020-01-27 12:06:50,250] INFO - Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.6.1->-r /opt/program/requirements.txt (line 1)) (2.8)

/[2020-01-27 12:06:50,433] INFO - Collecting pytz>=2017.2

[2020-01-27 12:06:50,458] INFO -   Downloading https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl (509kB)

\[2020-01-27 12:06:50,638] INFO - Collecting MarkupSafe>=0.9.2

[2020-01-27 12:06:50,657] INFO -   Downloading https://files.pythonhosted.org/packages/98/7b/ff284bd8c80654e471b769062a9b43cc5d03e7a615048d96f4619df8d420/MarkupSafe-1.1.1-cp37-cp37m-manylinux1_x86_64.whl

-[2020-01-27 12:06:50,725] INFO - Collecting docutils<0.16,>=0.10

[2020-01-27 12:06:50,747] INFO -   Downloading https://files.pythonhosted.org/packages/22/cd/a6aa959dca619918ccb55023b4cb151949c64d4d5d55b3f4ffd7eee0c6e8/docutils-0.15.2-py3-none-any.whl (547kB)

/[2020-01-27 12:06:50,828] INFO - Building wheels for collected packages: alembic, cerberus, python-json-logger, tabulate, sqlalchemy, prometheus-client, Mako

[2020-01-27 12:06:50,829] INFO -   Building wheel for alembic (setup.py): started

/[2020-01-27 12:06:51,238] INFO -   Building wheel for alembic (setup.py): finished with status 'done'

[2020-01-27 12:06:51,244] INFO -   Created wheel for alembic: filename=alembic-1.3.3-py2.py3-none-any.whl size=155684 sha256=76a5b093519ce57d526ba29cc06848485685312b9b4cf9c9c2c831d01d6ed878
  Stored in directory: /root/.cache/pip/wheels/12/6a/49/94a4af65af2edbccf9f467c0f4b26f24d0e5b61be0d8a7e066

[2020-01-27 12:06:51,270] INFO -   Building wheel for cerberus (setup.py): started

-[2020-01-27 12:06:51,552] INFO -   Building wheel for cerberus (setup.py): finished with status 'done'

[2020-01-27 12:06:51,554] INFO -   Created wheel for cerberus: filename=Cerberus-1.3.2-cp37-none-any.whl size=54336 sha256=6c602bbd6a2ef588f7cc64cdb1962fd18d770148be7866e9e67a3aa828c3eb4a

[2020-01-27 12:06:51,554] INFO -   Stored in directory: /root/.cache/pip/wheels/e9/38/1f/f2cc84182676f3ae7134b9b2d744f9c235b24d2ddc8f7fe465

[2020-01-27 12:06:51,566] INFO -   Building wheel for python-json-logger (setup.py): started

\[2020-01-27 12:06:51,830] INFO -   Building wheel for python-json-logger (setup.py): finished with status 'done'
  Created wheel for python-json-logger: filename=python_json_logger-0.1.11-py2.py3-none-any.whl size=5076 sha256=3caa5cfdc376c76780f99612bcdb4b589b30a0bab700e629c32be1968b3a8a2a
  Stored in directory: /root/.cache/pip/wheels/97/f7/a1/752e22bb30c1cfe38194ea0070a5c66e76ef4d06ad0c7dc401
  Building wheel for tabulate (setup.py): started

/[2020-01-27 12:06:52,100] INFO -   Building wheel for tabulate (setup.py): finished with status 'done'

[2020-01-27 12:06:52,101] INFO -   Created wheel for tabulate: filename=tabulate-0.8.6-cp37-none-any.whl size=23274 sha256=cc12543e51b02de4ed78e669430e0c6ad9d70310d1dd213c56bea0c9391ad228

[2020-01-27 12:06:52,101] INFO -   Stored in directory: /root/.cache/pip/wheels/9c/9b/f4/eb243fdb89676ec00588e8c54bb54360724c06e7fafe95278e

[2020-01-27 12:06:52,108] INFO -   Building wheel for sqlalchemy (setup.py): started

/[2020-01-27 12:06:53,719] INFO -   Building wheel for sqlalchemy (setup.py): finished with status 'done'

[2020-01-27 12:06:53,725] INFO -   Created wheel for sqlalchemy: filename=SQLAlchemy-1.3.13-cp37-cp37m-linux_x86_64.whl size=1223695 sha256=28d87592c11bbd7c385641828a6a773c88dfcb93e24263e9e7eecf6a046080cd
  Stored in directory: /root/.cache/pip/wheels/b3/35/98/4c9cb3fd63d21d5606b972dd70643769745adf60e622467b71

|[2020-01-27 12:06:53,835] INFO -   Building wheel for prometheus-client (setup.py): started

/[2020-01-27 12:06:54,132] INFO -   Building wheel for prometheus-client (setup.py): finished with status 'done'

[2020-01-27 12:06:54,133] INFO -   Created wheel for prometheus-client: filename=prometheus_client-0.7.1-cp37-none-any.whl size=41402 sha256=88e583efe8010312d83c8382121095f62f49a321e6c98e14b4a9f70d4cb422a3
  Stored in directory: /root/.cache/pip/wheels/1c/54/34/fd47cd9b308826cc4292b54449c1899a30251ef3b506bc91ea

[2020-01-27 12:06:54,144] INFO -   Building wheel for Mako (setup.py): started

/[2020-01-27 12:06:54,515] INFO -   Building wheel for Mako (setup.py): finished with status 'done'

[2020-01-27 12:06:54,516] INFO -   Created wheel for Mako: filename=Mako-1.1.1-cp37-none-any.whl size=75411 sha256=0b031671a27f16c591528c5181a6e49a2dcfabcd4f33bc6db05ae334b99a2603
  Stored in directory: /root/.cache/pip/wheels/ff/50/a9/0ddeed8679a1fb65bf4677cb9c92701828b2c1821e22ef72fd

[2020-01-27 12:06:54,533] INFO - Successfully built alembic cerberus python-json-logger tabulate sqlalchemy prometheus-client Mako

-[2020-01-27 12:06:54,901] INFO - Installing collected packages: websocket-client, docker, sqlalchemy, MarkupSafe, Mako, python-editor, python-dateutil, alembic, ruamel.yaml.clib, ruamel.yaml, cerberus, Werkzeug, click, itsdangerous, Jinja2, flask, python-json-logger, tabulate, protobuf, configparser, grpcio, docutils, jmespath, botocore, s3transfer, boto3, pyparsing, packaging, prometheus-client, humanfriendly, pytz, pandas, bentoml, torch, pillow, torchvision, joblib, scikit-learn, imageio

-[2020-01-27 12:07:16,337] INFO - Successfully installed Jinja2-2.11.0 Mako-1.1.1 MarkupSafe-1.1.1 Werkzeug-0.16.1 alembic-1.3.3 bentoml-0.6.1 boto3-1.11.9 botocore-1.14.9 cerberus-1.3.2 click-7.0 configparser-4.0.2 docker-4.1.0 docutils-0.15.2 flask-1.1.1 grpcio-1.26.0 humanfriendly-4.18 imageio-2.6.1 itsdangerous-1.1.0 jmespath-0.9.4 joblib-0.14.1 packaging-20.1 pandas-0.25.3 pillow-7.0.0 prometheus-client-0.7.1 protobuf-3.11.2 pyparsing-2.4.6 python-dateutil-2.8.0 python-editor-1.0.4 python-json-logger-0.1.11 pytz-2019.3 ruamel.yaml-0.16.6 ruamel.yaml.clib-0.2.0 s3transfer-0.3.2 scikit-learn-0.22.1 sqlalchemy-1.3.13 tabulate-0.8.6 torch-1.4.0 torchvision-0.5.0 websocket-client-0.57.0

|[2020-01-27 12:07:52,889] INFO -  ---> 0b1ebe21b3e4

[2020-01-27 12:07:52,890] INFO - Step 9/11 : RUN if [ -f /bento/bentoml_init.sh ]; then /bin/bash -c /bento/bentoml_init.sh; fi
[2020-01-27 12:07:52,890] INFO - 

-[2020-01-27 12:07:53,021] INFO -  ---> Running in 2a00d75527ea

|[2020-01-27 12:07:54,463] INFO -  ---> e63414b3d60f

[2020-01-27 12:07:54,463] INFO - Step 10/11 : RUN if [ -f /opt/program/setup.sh ]; then /bin/bash -c /opt/program/setup.sh; fi
[2020-01-27 12:07:54,463] INFO - 

\[2020-01-27 12:07:54,639] INFO -  ---> Running in b9f0b9484b33

\[2020-01-27 12:07:56,300] INFO -  ---> 8b42fcfaf6e5

[2020-01-27 12:07:56,300] INFO - Step 11/11 : ENV PATH="/opt/program:${PATH}"
[2020-01-27 12:07:56,300] INFO - 

-[2020-01-27 12:07:56,410] INFO -  ---> Running in ba8aa95b4b64

|[2020-01-27 12:07:56,620] INFO -  ---> 034276b99746

\[2020-01-27 12:07:56,632] INFO - Successfully built 034276b99746

[2020-01-27 12:07:56,636] INFO - Successfully tagged 192023623294.dkr.ecr.us-west-2.amazonaws.com/pytorchimageclassifier-sagemaker:20200127115720_71B982

/[2020-01-27 12:19:11,763] INFO - ApplyDeployment (first-pytorch, namespace dev) succeeded
Successfully created AWS Sagemaker deployment first-pytorch
{
  "namespace": "dev",
  "name": "first-pytorch",
  "spec": {
    "bentoName": "PytorchImageClassifier",
    "bentoVersion": "20200127115720_71B982",
    "operator": "AWS_SAGEMAKER",
    "sagemakerOperatorConfig": {
      "region": "us-west-2",
      "instanceType": "ml.m4.xlarge",
      "instanceCount": 1,
      "apiName": "predict"
    }
  },
  "state": {
    "state": "RUNNING",
    "infoJson": {
      "EndpointName": "dev-first-pytorch",
      "EndpointArn": "arn:aws:sagemaker:us-west-2:192023623294:endpoint/dev-first-pytorch",
      "EndpointConfigName": "dev-first-pytorch-PytorchImageClassifier-20200127115720-71B982",
      "ProductionVariants": [
        {
          "VariantName": "dev-first-pytorch-PytorchImageClassifier-20200127115720-71B982",
          "DeployedImages": [
            {
              "SpecifiedImage": "192023623294.dkr.ecr.us-west-2.amazonaws.com/pytorchimageclassifier-sagemaker:20200127115720_71B982",
              "ResolvedImage": "192023623294.dkr.ecr.us-west-2.amazonaws.com/[email protected]:60c5dccbc2fd0ea0eccf88564539f6c3b34d1f8b118466d14c16c30c4a7f5268",
              "ResolutionTime": "2020-01-27 12:19:13.669000-08:00"
            }
          ],
          "CurrentWeight": 1.0,
          "DesiredWeight": 1.0,
          "CurrentInstanceCount": 1,
          "DesiredInstanceCount": 1
        }
      ],
      "EndpointStatus": "InService",
      "CreationTime": "2020-01-27 12:19:11.719000-08:00",
      "LastModifiedTime": "2020-01-27 12:28:24.976000-08:00",
      "ResponseMetadata": {
        "RequestId": "42db6ea8-ecf9-4a39-9e9e-bfca9340104e",
        "HTTPStatusCode": 200,
        "HTTPHeaders": {
          "x-amzn-requestid": "42db6ea8-ecf9-4a39-9e9e-bfca9340104e",
          "content-type": "application/x-amz-json-1.1",
          "content-length": "839",
          "date": "Mon, 27 Jan 2020 20:28:25 GMT"
        },
        "RetryAttempts": 0
      }
    },
    "timestamp": "2020-01-27T20:28:26.011116Z"
  },
  "createdAt": "2020-01-27T20:05:28.156619Z",
  "lastUpdatedAt": "2020-01-27T20:05:28.156658Z"
}

We can use bentoml sagemaker get <deployment name> to get the latest information

In [23]:
!bentoml sagemaker get first-pytorch

{
  "namespace": "dev",
  "name": "first-pytorch",
  "spec": {
    "bentoName": "PytorchImageClassifier",
    "bentoVersion": "20200127115720_71B982",
    "operator": "AWS_SAGEMAKER",
    "sagemakerOperatorConfig": {
      "region": "us-west-2",
      "instanceType": "ml.m4.xlarge",
      "instanceCount": 1,
      "apiName": "predict"
    }
  },
  "state": {
    "state": "RUNNING",
    "infoJson": {
      "EndpointName": "dev-first-pytorch",
      "EndpointArn": "arn:aws:sagemaker:us-west-2:192023623294:endpoint/dev-first-pytorch",
      "EndpointConfigName": "dev-first-pytorch-PytorchImageClassifier-20200127115720-71B982",
      "ProductionVariants": [
        {
          "VariantName": "dev-first-pytorch-PytorchImageClassifier-20200127115720-71B982",
          "DeployedImages": [
            {
              "SpecifiedImage": "192023623294.dkr.ecr.us-west-2.amazonaws.com/pytorchimageclassifier-sagemaker:20200127115720_71B982",
              "ResolvedImage": "192023623294.dkr.ecr.us-west-2.amazonaws.com/[email protected]:60c5dccbc2fd0ea0eccf88564539f6c3b34d1f8b118466d14c16c30c4a7f5268",
              "ResolutionTime": "2020-01-27 12:19:13.669000-08:00"
            }
          ],
          "CurrentWeight": 1.0,
          "DesiredWeight": 1.0,
          "CurrentInstanceCount": 1,
          "DesiredInstanceCount": 1
        }
      ],
      "EndpointStatus": "InService",
      "CreationTime": "2020-01-27 12:19:11.719000-08:00",
      "LastModifiedTime": "2020-01-27 12:28:24.976000-08:00",
      "ResponseMetadata": {
        "RequestId": "63ccfe2d-bc46-41fc-afb8-03469e14dbd9",
        "HTTPStatusCode": 200,
        "HTTPHeaders": {
          "x-amzn-requestid": "63ccfe2d-bc46-41fc-afb8-03469e14dbd9",
          "content-type": "application/x-amz-json-1.1",
          "content-length": "839",
          "date": "Mon, 27 Jan 2020 20:29:47 GMT"
        },
        "RetryAttempts": 0
      }
    },
    "timestamp": "2020-01-27T20:29:47.288957Z"
  },
  "createdAt": "2020-01-27T20:05:28.156619Z",
  "lastUpdatedAt": "2020-01-27T20:05:28.156658Z"
}

Quick test with sample data to validate prediction result from the deployment

In [4]:
!aws sagemaker-runtime invoke-endpoint --endpoint-name dev-first-pytorch \
--body fileb:///Users/bozhaoyu/src/bento_gallery/pytorch/cifar10-image-classification/bird.jpg \
--content-type image/jpg output.json && cat output.json

{
    "ContentType": "application/json",
    "InvokedProductionVariant": "dev-first-pytorch-PytorchImageClassifier-20200127115720-71B982"
}
"bird"

bentoml sagemaker list is great tool to list all of the sagemaker dpeloyments

In [5]:
!bentoml sagemaker list

NAME           NAMESPACE    LABELS    PLATFORM       STATUS    AGE
first-pytorch  dev                    aws-sagemaker  running   38 minutes and 11.5 seconds

bentoml sagemaker delete delete Sagemaker deployment and its related resources

In [6]:
!bentoml sagemaker delete first-pytorch

Successfully deleted AWS Sagemaker deployment "first-pytorch"
In [ ]: