BentoML Example: Keras Toxic Comment Classification

BentoML is an open source platform for machine learning model serving and deployment.

This notebook demonstrates how to use BentoML to turn a Keras model into a docker image containing a REST API server serving this model, how to use your ML service built with BentoML as a CLI tool, and how to distribute it a pypi package.

This notebook is built based on: https://www.kaggle.com/sarvajna/keras-sequential-model-lb-0-052

Impression

In [4]:
%reload_ext autoreload
%autoreload 2
%matplotlib inline
In [ ]:
!pip install bentoml
!pip install keras kaggle tensorflow==1.14.0
In [5]:
import bentoml
import numpy as np
import pandas as pd
from keras.preprocessing import text, sequence
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation
from keras.layers import Embedding
from keras.layers import Conv1D, GlobalMaxPooling1D
from sklearn.model_selection import train_test_split
Using TensorFlow backend.
In [6]:
list_of_classes = ["toxic", "severe_toxic", "obscene", "threat", "insult", "identity_hate"]
max_features = 20000
max_text_length = 400
embedding_dims = 50
filters = 250
kernel_size = 3
hidden_dims = 250
batch_size = 32
epochs = 2

Prepare Dataset

Please Download data with Kaggle at https://www.kaggle.com/sarvajna/keras-sequential-model-lb-0-052/data

If you are running this notebook in Google Colab, fill in your kaggle credential below and download the training dataset from Kaggle:

In [7]:
%%bash

export KAGGLE_USERNAME=
export KAGGLE_KEY=

if [ ! -f ./train.csv.zip ]; then
    kaggle competitions download -c jigsaw-toxic-comment-classification-challenge
    unzip train.csv.zip
    unzip sample_submission.csv.zip
    unzip test.csv.zip
    unzip test_labels.csv.zip
fi
In [8]:
train_df = pd.read_csv('./train.csv')

print(train_df.head())
                 id                                       comment_text  toxic  \
0  0000997932d777bf  Explanation\nWhy the edits made under my usern...      0   
1  000103f0d9cfb60f  D'aww! He matches this background colour I'm s...      0   
2  000113f07ec002fd  Hey man, I'm really not trying to edit war. It...      0   
3  0001b41b1c6bb37e  "\nMore\nI can't make any real suggestions on ...      0   
4  0001d958c54c6e35  You, sir, are my hero. Any chance you remember...      0   

   severe_toxic  obscene  threat  insult  identity_hate  
0             0        0       0       0              0  
1             0        0       0       0              0  
2             0        0       0       0              0  
3             0        0       0       0              0  
4             0        0       0       0              0  
In [9]:
x = train_df['comment_text'].values
print(x)
["Explanation\nWhy the edits made under my username Hardcore Metallica Fan were reverted? They weren't vandalisms, just closure on some GAs after I voted at New York Dolls FAC. And please don't remove the template from the talk page since I'm retired now.89.205.38.27"
 "D'aww! He matches this background colour I'm seemingly stuck with. Thanks.  (talk) 21:51, January 11, 2016 (UTC)"
 "Hey man, I'm really not trying to edit war. It's just that this guy is constantly removing relevant information and talking to me through edits instead of my talk page. He seems to care more about the formatting than the actual info."
 ...
 'Spitzer \n\nUmm, theres no actual article for prostitution ring.  - Crunch Captain.'
 'And it looks like it was actually you who put on the speedy to have the first version deleted now that I look at it.'
 '"\nAnd ... I really don\'t think you understand.  I came here and my idea was bad right away.  What kind of community goes ""you have bad ideas"" go away, instead of helping rewrite them.   "']
In [10]:
y = train_df[list_of_classes].values
print(y)
[[0 0 0 0 0 0]
 [0 0 0 0 0 0]
 [0 0 0 0 0 0]
 ...
 [0 0 0 0 0 0]
 [0 0 0 0 0 0]
 [0 0 0 0 0 0]]
In [11]:
x_tokenizer = text.Tokenizer(num_words=max_features)
print(x_tokenizer)
x_tokenizer.fit_on_texts(list(x))
print(x_tokenizer)
x_tokenized = x_tokenizer.texts_to_sequences(x) #list of lists(containing numbers), so basically a list of sequences, not a numpy array
#pad_sequences:transform a list of num_samples sequences (lists of scalars) into a 2D Numpy array of shape 
x_train_val = sequence.pad_sequences(x_tokenized, maxlen=max_text_length)
<keras_preprocessing.text.Tokenizer object at 0x1092bbac8>
<keras_preprocessing.text.Tokenizer object at 0x1092bbac8>
In [12]:
x_train, x_val, y_train, y_val = train_test_split(x_train_val, y, test_size=0.1, random_state=1)
In [13]:
print('Build model...')
model = Sequential()

# we start off with an efficient embedding layer which maps
# our vocab indices into embedding_dims dimensions
model.add(Embedding(max_features,
                    embedding_dims,
                    input_length=max_text_length))
model.add(Dropout(0.2))

# we add a Convolution1D, which will learn filters
# word group filters of size filter_length:
model.add(Conv1D(filters,
                 kernel_size,
                 padding='valid',
                 activation='relu',
                 strides=1))
# we use max pooling:
model.add(GlobalMaxPooling1D())

# We add a vanilla hidden layer:
model.add(Dense(hidden_dims))
model.add(Dropout(0.2))
model.add(Activation('relu'))

# We project onto 6 output layers, and squash it with a sigmoid:
model.add(Dense(6))
model.add(Activation('sigmoid'))

model.compile(loss='binary_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])

model.summary()
WARNING: Logging before flag parsing goes to stderr.
W0213 14:05:45.043107 4456553920 deprecation_wrapper.py:119] From /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:74: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.

W0213 14:05:45.058714 4456553920 deprecation_wrapper.py:119] From /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

W0213 14:05:45.061474 4456553920 deprecation_wrapper.py:119] From /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.

W0213 14:05:45.076580 4456553920 deprecation_wrapper.py:119] From /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:133: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.

W0213 14:05:45.085232 4456553920 deprecation.py:506] From /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
W0213 14:05:45.151792 4456553920 deprecation_wrapper.py:119] From /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages/keras/optimizers.py:790: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.

W0213 14:05:45.168698 4456553920 deprecation_wrapper.py:119] From /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:3376: The name tf.log is deprecated. Please use tf.math.log instead.

W0213 14:05:45.173859 4456553920 deprecation.py:323] From /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages/tensorflow/python/ops/nn_impl.py:180: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
Build model...
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_1 (Embedding)      (None, 400, 50)           1000000   
_________________________________________________________________
dropout_1 (Dropout)          (None, 400, 50)           0         
_________________________________________________________________
conv1d_1 (Conv1D)            (None, 398, 250)          37750     
_________________________________________________________________
global_max_pooling1d_1 (Glob (None, 250)               0         
_________________________________________________________________
dense_1 (Dense)              (None, 250)               62750     
_________________________________________________________________
dropout_2 (Dropout)          (None, 250)               0         
_________________________________________________________________
activation_1 (Activation)    (None, 250)               0         
_________________________________________________________________
dense_2 (Dense)              (None, 6)                 1506      
_________________________________________________________________
activation_2 (Activation)    (None, 6)                 0         
=================================================================
Total params: 1,102,006
Trainable params: 1,102,006
Non-trainable params: 0
_________________________________________________________________
In [14]:
model.fit(x_train, y_train,
          batch_size=batch_size,
          epochs=epochs,
validation_data=(x_val, y_val))
Train on 143613 samples, validate on 15958 samples
Epoch 1/2
143613/143613 [==============================] - 248s 2ms/step - loss: 0.0631 - acc: 0.9787 - val_loss: 0.0529 - val_acc: 0.9808
Epoch 2/2
143613/143613 [==============================] - 250s 2ms/step - loss: 0.0463 - acc: 0.9828 - val_loss: 0.0509 - val_acc: 0.9812
Out[14]:
<keras.callbacks.History at 0x13d120940>
In [15]:
test_df = pd.read_csv('./test.csv')
In [16]:
x_test = test_df['comment_text'].values
In [17]:
x_test_tokenized = x_tokenizer.texts_to_sequences(x_test)
x_testing = sequence.pad_sequences(x_test_tokenized, maxlen=max_text_length)
In [18]:
y_testing = model.predict(x_testing, verbose = 1)
153164/153164 [==============================] - 43s 283us/step
In [19]:
sample_submission = pd.read_csv("./sample_submission.csv")
sample_submission[list_of_classes] = y_testing
sample_submission.to_csv("toxic_comment_classification.csv", index=False)

Create BentoService for model serving

In [20]:
%%writefile toxic_comment_classifier.py

from bentoml import api, artifacts, env, BentoService
from bentoml.artifact import PickleArtifact, KerasModelArtifact
from bentoml.handlers import DataframeHandler

from keras.preprocessing import text, sequence
import numpy as np

list_of_classes = ["toxic", "severe_toxic", "obscene", "threat", "insult", "identity_hate"]
max_text_length = 400

@env(pip_dependencies=['tensorflow==1.14.0', 'keras', 'pandas', 'numpy'])
@artifacts([PickleArtifact('x_tokenizer'), KerasModelArtifact('model')])
class ToxicCommentClassification(BentoService):
    
    def tokenize_df(self, df):
        comments = df['comment_text'].values
        tokenized = self.artifacts.x_tokenizer.texts_to_sequences(comments)        
        input_data = sequence.pad_sequences(tokenized, maxlen=max_text_length)
        return input_data
    
    @api(DataframeHandler)
    def predict(self, df):
        input_data = self.tokenize_df(df)
        prediction = self.artifacts.model.predict(input_data)
        result = []
        for i in prediction:
            result.append(list_of_classes[np.argmax(i)])
        return result
Overwriting toxic_comment_classifier.py

Save BentoService to file archive

In [21]:
# 1) import the custom BentoService defined above
from toxic_comment_classifier import ToxicCommentClassification

# 2) `pack` it with required artifacts
svc = ToxicCommentClassification()
svc.pack('x_tokenizer', x_tokenizer)
svc.pack('model', model)

# 3) save your BentoSerivce
saved_path = svc.save()
[2020-02-13 14:15:03,839] WARNING - BentoML local changes detected - Local BentoML repository including all code changes will be bundled together with the BentoService bundle. When used with docker, the base docker image will be default to same version as last PyPI release at version: 0.6.2. You can also force bentoml to use a specific version for deploying your BentoService bundle, by setting the config 'core/bentoml_deploy_version' to a pinned version or your custom BentoML on github, e.g.:'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
W0213 14:15:03.842620 4456553920 deprecation_wrapper.py:119] From /Users/bozhaoyu/src/bento/bentoml/artifact/keras_model_artifact.py:100: The name tf.keras.backend.get_session is deprecated. Please use tf.compat.v1.keras.backend.get_session instead.

[2020-02-13 14:15:04,172] WARNING - BentoML local changes detected - Local BentoML repository including all code changes will be bundled together with the BentoService bundle. When used with docker, the base docker image will be default to same version as last PyPI release at version: 0.6.2. You can also force bentoml to use a specific version for deploying your BentoService bundle, by setting the config 'core/bentoml_deploy_version' to a pinned version or your custom BentoML on github, e.g.:'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
[2020-02-13 14:15:35,253] WARNING - BentoML local changes detected - Local BentoML repository including all code changes will be bundled together with the BentoService bundle. When used with docker, the base docker image will be default to same version as last PyPI release at version: 0.6.2. You can also force bentoml to use a specific version for deploying your BentoService bundle, by setting the config 'core/bentoml_deploy_version' to a pinned version or your custom BentoML on github, e.g.:'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
running sdist
running egg_info
writing BentoML.egg-info/PKG-INFO
writing dependency_links to BentoML.egg-info/dependency_links.txt
writing entry points to BentoML.egg-info/entry_points.txt
writing requirements to BentoML.egg-info/requires.txt
writing top-level names to BentoML.egg-info/top_level.txt
reading manifest file 'BentoML.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching 'examples'
no previously-included directories found matching 'tests'
no previously-included directories found matching 'docs'
no previously-included directories found matching 'scripts'
warning: no previously-included files matching '*~' found anywhere in distribution
warning: no previously-included files matching '*.pyo' found anywhere in distribution
warning: no previously-included files matching '.git' found anywhere in distribution
warning: no previously-included files matching '.ipynb_checkpoints' found anywhere in distribution
warning: no previously-included files matching '__pycache__' found anywhere in distribution
writing manifest file 'BentoML.egg-info/SOURCES.txt'
running check
warning: check: missing meta-data: if 'author' supplied, 'author_email' must be supplied too

creating BentoML-0.6.2+8.gd95a887
creating BentoML-0.6.2+8.gd95a887/BentoML.egg-info
creating BentoML-0.6.2+8.gd95a887/bentoml
creating BentoML-0.6.2+8.gd95a887/bentoml/artifact
creating BentoML-0.6.2+8.gd95a887/bentoml/bundler
creating BentoML-0.6.2+8.gd95a887/bentoml/cli
creating BentoML-0.6.2+8.gd95a887/bentoml/clipper
creating BentoML-0.6.2+8.gd95a887/bentoml/configuration
creating BentoML-0.6.2+8.gd95a887/bentoml/deployment
creating BentoML-0.6.2+8.gd95a887/bentoml/deployment/aws_lambda
creating BentoML-0.6.2+8.gd95a887/bentoml/deployment/sagemaker
creating BentoML-0.6.2+8.gd95a887/bentoml/handlers
creating BentoML-0.6.2+8.gd95a887/bentoml/marshal
creating BentoML-0.6.2+8.gd95a887/bentoml/migrations
creating BentoML-0.6.2+8.gd95a887/bentoml/migrations/versions
creating BentoML-0.6.2+8.gd95a887/bentoml/proto
creating BentoML-0.6.2+8.gd95a887/bentoml/repository
creating BentoML-0.6.2+8.gd95a887/bentoml/server
creating BentoML-0.6.2+8.gd95a887/bentoml/server/static
creating BentoML-0.6.2+8.gd95a887/bentoml/utils
creating BentoML-0.6.2+8.gd95a887/bentoml/utils/validator
creating BentoML-0.6.2+8.gd95a887/bentoml/yatai
creating BentoML-0.6.2+8.gd95a887/bentoml/yatai/client
copying files to BentoML-0.6.2+8.gd95a887...
copying LICENSE -> BentoML-0.6.2+8.gd95a887
copying MANIFEST.in -> BentoML-0.6.2+8.gd95a887
copying README.md -> BentoML-0.6.2+8.gd95a887
copying pyproject.toml -> BentoML-0.6.2+8.gd95a887
copying setup.cfg -> BentoML-0.6.2+8.gd95a887
copying setup.py -> BentoML-0.6.2+8.gd95a887
copying versioneer.py -> BentoML-0.6.2+8.gd95a887
copying BentoML.egg-info/PKG-INFO -> BentoML-0.6.2+8.gd95a887/BentoML.egg-info
copying BentoML.egg-info/SOURCES.txt -> BentoML-0.6.2+8.gd95a887/BentoML.egg-info
copying BentoML.egg-info/dependency_links.txt -> BentoML-0.6.2+8.gd95a887/BentoML.egg-info
copying BentoML.egg-info/entry_points.txt -> BentoML-0.6.2+8.gd95a887/BentoML.egg-info
copying BentoML.egg-info/requires.txt -> BentoML-0.6.2+8.gd95a887/BentoML.egg-info
copying BentoML.egg-info/top_level.txt -> BentoML-0.6.2+8.gd95a887/BentoML.egg-info
copying bentoml/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml
copying bentoml/_version.py -> BentoML-0.6.2+8.gd95a887/bentoml
copying bentoml/alembic.ini -> BentoML-0.6.2+8.gd95a887/bentoml
copying bentoml/db.py -> BentoML-0.6.2+8.gd95a887/bentoml
copying bentoml/exceptions.py -> BentoML-0.6.2+8.gd95a887/bentoml
copying bentoml/service.py -> BentoML-0.6.2+8.gd95a887/bentoml
copying bentoml/service_env.py -> BentoML-0.6.2+8.gd95a887/bentoml
copying bentoml/artifact/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/fastai_model_artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/h2o_model_artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/keras_model_artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/lightgbm_model_artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/pickle_artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/pytorch_model_artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/sklearn_model_artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/text_file_artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/tf_savedmodel_artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/artifact/xgboost_model_artifact.py -> BentoML-0.6.2+8.gd95a887/bentoml/artifact
copying bentoml/bundler/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/bundler
copying bentoml/bundler/bundler.py -> BentoML-0.6.2+8.gd95a887/bentoml/bundler
copying bentoml/bundler/config.py -> BentoML-0.6.2+8.gd95a887/bentoml/bundler
copying bentoml/bundler/loader.py -> BentoML-0.6.2+8.gd95a887/bentoml/bundler
copying bentoml/bundler/py_module_utils.py -> BentoML-0.6.2+8.gd95a887/bentoml/bundler
copying bentoml/bundler/templates.py -> BentoML-0.6.2+8.gd95a887/bentoml/bundler
copying bentoml/bundler/utils.py -> BentoML-0.6.2+8.gd95a887/bentoml/bundler
copying bentoml/cli/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/cli
copying bentoml/cli/aws_lambda.py -> BentoML-0.6.2+8.gd95a887/bentoml/cli
copying bentoml/cli/aws_sagemaker.py -> BentoML-0.6.2+8.gd95a887/bentoml/cli
copying bentoml/cli/bento.py -> BentoML-0.6.2+8.gd95a887/bentoml/cli
copying bentoml/cli/click_utils.py -> BentoML-0.6.2+8.gd95a887/bentoml/cli
copying bentoml/cli/config.py -> BentoML-0.6.2+8.gd95a887/bentoml/cli
copying bentoml/cli/deployment.py -> BentoML-0.6.2+8.gd95a887/bentoml/cli
copying bentoml/cli/utils.py -> BentoML-0.6.2+8.gd95a887/bentoml/cli
copying bentoml/clipper/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/clipper
copying bentoml/configuration/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/configuration
copying bentoml/configuration/configparser.py -> BentoML-0.6.2+8.gd95a887/bentoml/configuration
copying bentoml/configuration/default_bentoml.cfg -> BentoML-0.6.2+8.gd95a887/bentoml/configuration
copying bentoml/deployment/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment
copying bentoml/deployment/operator.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment
copying bentoml/deployment/store.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment
copying bentoml/deployment/utils.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment
copying bentoml/deployment/aws_lambda/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment/aws_lambda
copying bentoml/deployment/aws_lambda/download_extra_resources.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment/aws_lambda
copying bentoml/deployment/aws_lambda/lambda_app.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment/aws_lambda
copying bentoml/deployment/aws_lambda/utils.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment/aws_lambda
copying bentoml/deployment/sagemaker/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment/sagemaker
copying bentoml/deployment/sagemaker/sagemaker_nginx.conf -> BentoML-0.6.2+8.gd95a887/bentoml/deployment/sagemaker
copying bentoml/deployment/sagemaker/sagemaker_serve.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment/sagemaker
copying bentoml/deployment/sagemaker/sagemaker_wsgi.py -> BentoML-0.6.2+8.gd95a887/bentoml/deployment/sagemaker
copying bentoml/handlers/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/handlers
copying bentoml/handlers/base_handlers.py -> BentoML-0.6.2+8.gd95a887/bentoml/handlers
copying bentoml/handlers/clipper_handler.py -> BentoML-0.6.2+8.gd95a887/bentoml/handlers
copying bentoml/handlers/dataframe_handler.py -> BentoML-0.6.2+8.gd95a887/bentoml/handlers
copying bentoml/handlers/fastai_image_handler.py -> BentoML-0.6.2+8.gd95a887/bentoml/handlers
copying bentoml/handlers/image_handler.py -> BentoML-0.6.2+8.gd95a887/bentoml/handlers
copying bentoml/handlers/json_handler.py -> BentoML-0.6.2+8.gd95a887/bentoml/handlers
copying bentoml/handlers/pytorch_tensor_handler.py -> BentoML-0.6.2+8.gd95a887/bentoml/handlers
copying bentoml/handlers/tensorflow_tensor_handler.py -> BentoML-0.6.2+8.gd95a887/bentoml/handlers
copying bentoml/handlers/utils.py -> BentoML-0.6.2+8.gd95a887/bentoml/handlers
copying bentoml/marshal/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/marshal
copying bentoml/marshal/marshal.py -> BentoML-0.6.2+8.gd95a887/bentoml/marshal
copying bentoml/marshal/utils.py -> BentoML-0.6.2+8.gd95a887/bentoml/marshal
copying bentoml/migrations/README -> BentoML-0.6.2+8.gd95a887/bentoml/migrations
copying bentoml/migrations/env.py -> BentoML-0.6.2+8.gd95a887/bentoml/migrations
copying bentoml/migrations/script.py.mako -> BentoML-0.6.2+8.gd95a887/bentoml/migrations
copying bentoml/migrations/versions/a6b00ae45279_add_last_updated_at_for_deployments.py -> BentoML-0.6.2+8.gd95a887/bentoml/migrations/versions
copying bentoml/proto/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/proto
copying bentoml/proto/deployment_pb2.py -> BentoML-0.6.2+8.gd95a887/bentoml/proto
copying bentoml/proto/repository_pb2.py -> BentoML-0.6.2+8.gd95a887/bentoml/proto
copying bentoml/proto/status_pb2.py -> BentoML-0.6.2+8.gd95a887/bentoml/proto
copying bentoml/proto/yatai_service_pb2.py -> BentoML-0.6.2+8.gd95a887/bentoml/proto
copying bentoml/proto/yatai_service_pb2_grpc.py -> BentoML-0.6.2+8.gd95a887/bentoml/proto
copying bentoml/repository/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/repository
copying bentoml/repository/metadata_store.py -> BentoML-0.6.2+8.gd95a887/bentoml/repository
copying bentoml/server/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/server
copying bentoml/server/bento_api_server.py -> BentoML-0.6.2+8.gd95a887/bentoml/server
copying bentoml/server/bento_sagemaker_server.py -> BentoML-0.6.2+8.gd95a887/bentoml/server
copying bentoml/server/gunicorn_config.py -> BentoML-0.6.2+8.gd95a887/bentoml/server
copying bentoml/server/gunicorn_server.py -> BentoML-0.6.2+8.gd95a887/bentoml/server
copying bentoml/server/marshal_server.py -> BentoML-0.6.2+8.gd95a887/bentoml/server
copying bentoml/server/middlewares.py -> BentoML-0.6.2+8.gd95a887/bentoml/server
copying bentoml/server/utils.py -> BentoML-0.6.2+8.gd95a887/bentoml/server
copying bentoml/server/static/swagger-ui-bundle.js -> BentoML-0.6.2+8.gd95a887/bentoml/server/static
copying bentoml/server/static/swagger-ui.css -> BentoML-0.6.2+8.gd95a887/bentoml/server/static
copying bentoml/utils/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/utils
copying bentoml/utils/cloudpickle.py -> BentoML-0.6.2+8.gd95a887/bentoml/utils
copying bentoml/utils/hybirdmethod.py -> BentoML-0.6.2+8.gd95a887/bentoml/utils
copying bentoml/utils/log.py -> BentoML-0.6.2+8.gd95a887/bentoml/utils
copying bentoml/utils/s3.py -> BentoML-0.6.2+8.gd95a887/bentoml/utils
copying bentoml/utils/tempdir.py -> BentoML-0.6.2+8.gd95a887/bentoml/utils
copying bentoml/utils/usage_stats.py -> BentoML-0.6.2+8.gd95a887/bentoml/utils
copying bentoml/utils/validator/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/utils/validator
copying bentoml/yatai/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/yatai
copying bentoml/yatai/deployment_utils.py -> BentoML-0.6.2+8.gd95a887/bentoml/yatai
copying bentoml/yatai/status.py -> BentoML-0.6.2+8.gd95a887/bentoml/yatai
copying bentoml/yatai/yatai_service_impl.py -> BentoML-0.6.2+8.gd95a887/bentoml/yatai
copying bentoml/yatai/client/__init__.py -> BentoML-0.6.2+8.gd95a887/bentoml/yatai/client
copying bentoml/yatai/client/bento_repository_api.py -> BentoML-0.6.2+8.gd95a887/bentoml/yatai/client
copying bentoml/yatai/client/deployment_api.py -> BentoML-0.6.2+8.gd95a887/bentoml/yatai/client
Writing BentoML-0.6.2+8.gd95a887/setup.cfg
UPDATING BentoML-0.6.2+8.gd95a887/bentoml/_version.py
set BentoML-0.6.2+8.gd95a887/bentoml/_version.py to '0.6.2+8.gd95a887'
Creating tar archive
removing 'BentoML-0.6.2+8.gd95a887' (and everything under it)
[2020-02-13 14:15:36,977] INFO - BentoService bundle 'ToxicCommentClassification:20200213141504_137C94' created at: /private/var/folders/kn/xnc9k74x03567n1mx2tfqnpr0000gn/T/bentoml-temp-n06n1a14
[2020-02-13 14:15:36,979] WARNING - BentoML local changes detected - Local BentoML repository including all code changes will be bundled together with the BentoService bundle. When used with docker, the base docker image will be default to same version as last PyPI release at version: 0.6.2. You can also force bentoml to use a specific version for deploying your BentoService bundle, by setting the config 'core/bentoml_deploy_version' to a pinned version or your custom BentoML on github, e.g.:'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
[2020-02-13 14:15:36,994] WARNING - Saved BentoService bundle version mismatch: loading BentoServie bundle create with BentoML version 0.6.2,  but loading from BentoML version 0.6.2+8.gd95a887
[2020-02-13 14:15:37,128] INFO - BentoService bundle 'ToxicCommentClassification:20200213141504_137C94' created at: /Users/bozhaoyu/bentoml/repository/ToxicCommentClassification/20200213141504_137C94

Load BentoService from archive

In [22]:
sample_test = test_df.iloc[40:42]
print(sample_test)
bento_service = bentoml.load(saved_path)

print(bento_service.predict(sample_test))
                  id                                       comment_text
40  0011cefc680993ba                      REDIRECT Talk:Mi Vida Eres TĂș
41  0011ef6aa33d42e6  " \n I'm not convinced that he was blind. Wher...
[2020-02-13 14:15:54,065] WARNING - BentoML local changes detected - Local BentoML repository including all code changes will be bundled together with the BentoService bundle. When used with docker, the base docker image will be default to same version as last PyPI release at version: 0.6.2. You can also force bentoml to use a specific version for deploying your BentoService bundle, by setting the config 'core/bentoml_deploy_version' to a pinned version or your custom BentoML on github, e.g.:'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
[2020-02-13 14:15:54,084] WARNING - Saved BentoService bundle version mismatch: loading BentoServie bundle create with BentoML version 0.6.2,  but loading from BentoML version 0.6.2+8.gd95a887
[2020-02-13 14:15:54,085] WARNING - Module `toxic_comment_classifier` already loaded, using existing imported module.
W0213 14:15:54.356201 4456553920 deprecation_wrapper.py:119] From /Users/bozhaoyu/src/bento/bentoml/artifact/keras_model_artifact.py:114: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead.

[2020-02-13 14:15:56,008] WARNING - BentoML local changes detected - Local BentoML repository including all code changes will be bundled together with the BentoService bundle. When used with docker, the base docker image will be default to same version as last PyPI release at version: 0.6.2. You can also force bentoml to use a specific version for deploying your BentoService bundle, by setting the config 'core/bentoml_deploy_version' to a pinned version or your custom BentoML on github, e.g.:'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
['toxic', 'toxic']
In [23]:
!bentoml get ToxicCommentClassification
BENTO_SERVICE                                     AGE            APIS                       ARTIFACTS
ToxicCommentClassification:20200213141504_137C94  26.84 seconds  predict<DataframeHandler>  x_tokenizer<PickleArtifact>, model<KerasModelArtifact>
In [24]:
!bentoml get ToxicCommentClassification:20200213141504_137C94 
{
  "name": "ToxicCommentClassification",
  "version": "20200213141504_137C94",
  "uri": {
    "type": "LOCAL",
    "uri": "/Users/bozhaoyu/bentoml/repository/ToxicCommentClassification/20200213141504_137C94"
  },
  "bentoServiceMetadata": {
    "name": "ToxicCommentClassification",
    "version": "20200213141504_137C94",
    "createdAt": "2020-02-13T22:15:35.254603Z",
    "env": {
      "condaEnv": "name: bentoml-ToxicCommentClassification\nchannels:\n- defaults\ndependencies:\n- python=3.7.3\n- pip\n",
      "pipDependencies": "bentoml==0.6.2\ntensorflow==1.14.0\nkeras\npandas\nnumpy",
      "pythonVersion": "3.7.3"
    },
    "artifacts": [
      {
        "name": "x_tokenizer",
        "artifactType": "PickleArtifact"
      },
      {
        "name": "model",
        "artifactType": "KerasModelArtifact"
      }
    ],
    "apis": [
      {
        "name": "predict",
        "handlerType": "DataframeHandler",
        "docs": "BentoService API",
        "handlerConfig": {
          "orient": "records",
          "typ": "frame",
          "input_dtypes": null,
          "output_orient": "records"
        }
      }
    ]
  }
}
In [25]:
!bentoml info ToxicCommentClassification:20200213141504_137C94 
[2020-02-13 14:16:23,638] WARNING - BentoML local changes detected - Local BentoML repository including all code changes will be bundled together with the BentoService bundle. When used with docker, the base docker image will be default to same version as last PyPI release at version: 0.6.2. You can also force bentoml to use a specific version for deploying your BentoService bundle, by setting the config 'core/bentoml_deploy_version' to a pinned version or your custom BentoML on github, e.g.:'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
[2020-02-13 14:16:23,664] WARNING - Saved BentoService bundle version mismatch: loading BentoServie bundle create with BentoML version 0.6.2,  but loading from BentoML version 0.6.2+8.gd95a887
{
  "name": "ToxicCommentClassification",
  "version": "20200213141504_137C94",
  "created_at": "2020-02-13T22:15:35.254603Z",
  "env": {
    "conda_env": "name: bentoml-ToxicCommentClassification\nchannels:\n- defaults\ndependencies:\n- python=3.7.3\n- pip\n",
    "pip_dependencies": "bentoml==0.6.2\ntensorflow==1.14.0\nkeras\npandas\nnumpy",
    "python_version": "3.7.3"
  },
  "artifacts": [
    {
      "name": "x_tokenizer",
      "artifact_type": "PickleArtifact"
    },
    {
      "name": "model",
      "artifact_type": "KerasModelArtifact"
    }
  ],
  "apis": [
    {
      "name": "predict",
      "handler_type": "DataframeHandler",
      "docs": "BentoService API",
      "handler_config": {
        "input_dtypes": null,
        "output_orient": "records",
        "orient": "records",
        "typ": "frame"
      }
    }
  ]
}
In [26]:
!bentoml run ToxicCommentClassification:20200213141504_137C94 predict --input '[{"comment_text": "bad terrible"}]'
[2020-02-13 14:16:34,090] WARNING - BentoML local changes detected - Local BentoML repository including all code changes will be bundled together with the BentoService bundle. When used with docker, the base docker image will be default to same version as last PyPI release at version: 0.6.2. You can also force bentoml to use a specific version for deploying your BentoService bundle, by setting the config 'core/bentoml_deploy_version' to a pinned version or your custom BentoML on github, e.g.:'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
[2020-02-13 14:16:34,112] WARNING - Saved BentoService bundle version mismatch: loading BentoServie bundle create with BentoML version 0.6.2,  but loading from BentoML version 0.6.2+8.gd95a887
Using TensorFlow backend.
[2020-02-13 14:16:37,621] WARNING - BentoML local changes detected - Local BentoML repository including all code changes will be bundled together with the BentoService bundle. When used with docker, the base docker image will be default to same version as last PyPI release at version: 0.6.2. You can also force bentoml to use a specific version for deploying your BentoService bundle, by setting the config 'core/bentoml_deploy_version' to a pinned version or your custom BentoML on github, e.g.:'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
2020-02-13 14:16:37.945167: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
WARNING: Logging before flag parsing goes to stderr.
W0213 14:16:37.945712 4525972928 deprecation_wrapper.py:119] From /Users/bozhaoyu/src/bento/bentoml/artifact/keras_model_artifact.py:114: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead.

W0213 14:16:37.958029 4525972928 deprecation_wrapper.py:119] From /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

W0213 14:16:37.959445 4525972928 deprecation_wrapper.py:119] From /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.

W0213 14:16:37.970021 4525972928 deprecation_wrapper.py:119] From /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:131: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.

W0213 14:16:37.970181 4525972928 deprecation_wrapper.py:119] From /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:133: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.

W0213 14:16:37.976413 4525972928 deprecation.py:506] From /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
W0213 14:16:38.074156 4525972928 deprecation_wrapper.py:119] From /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:174: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.

W0213 14:16:38.141179 4525972928 deprecation_wrapper.py:119] From /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages/keras/optimizers.py:790: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.

W0213 14:16:38.147212 4525972928 deprecation.py:323] From /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages/tensorflow/python/ops/nn_impl.py:180: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
W0213 14:16:38.886831 4525972928 deprecation_wrapper.py:119] From /Users/bozhaoyu/src/bento/bentoml/artifact/keras_model_artifact.py:100: The name tf.keras.backend.get_session is deprecated. Please use tf.compat.v1.keras.backend.get_session instead.

[2020-02-13 14:16:38,888] WARNING - BentoML local changes detected - Local BentoML repository including all code changes will be bundled together with the BentoService bundle. When used with docker, the base docker image will be default to same version as last PyPI release at version: 0.6.2. You can also force bentoml to use a specific version for deploying your BentoService bundle, by setting the config 'core/bentoml_deploy_version' to a pinned version or your custom BentoML on github, e.g.:'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
['toxic']

Use BentoService as PyPI package

In [29]:
!pip install {saved_path}
Processing /Users/bozhaoyu/bentoml/repository/ToxicCommentClassification/20200213131414_B4566D
Requirement already satisfied: bentoml==0.6.2 in /Users/bozhaoyu/src/bento (from ToxicCommentClassification===20200213131414-B4566D) (0.6.2+8.gd95a887)
Requirement already satisfied: tensorflow==1.14.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from ToxicCommentClassification===20200213131414-B4566D) (1.14.0)
Requirement already satisfied: keras in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from ToxicCommentClassification===20200213131414-B4566D) (2.2.4)
Requirement already satisfied: pandas in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from ToxicCommentClassification===20200213131414-B4566D) (0.24.2)
Requirement already satisfied: numpy in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from ToxicCommentClassification===20200213131414-B4566D) (1.16.4)
Requirement already satisfied: ruamel.yaml>=0.15.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (0.16.5)
Requirement already satisfied: flask in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (1.0.4)
Requirement already satisfied: gunicorn in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (19.9.0)
Requirement already satisfied: click>=7.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (7.0)
Requirement already satisfied: prometheus_client in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (0.7.1)
Requirement already satisfied: python-json-logger in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (0.1.11)
Requirement already satisfied: boto3 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (1.10.42)
Requirement already satisfied: requests in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (2.22.0)
Requirement already satisfied: packaging in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (19.0)
Requirement already satisfied: docker in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (4.1.0)
Requirement already satisfied: configparser in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (3.7.4)
Requirement already satisfied: sqlalchemy>=1.3.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (1.3.6)
Requirement already satisfied: protobuf>=3.6.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (3.7.0)
Requirement already satisfied: grpcio in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (1.22.0)
Requirement already satisfied: cerberus in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (1.3.1)
Requirement already satisfied: tabulate in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (0.8.3)
Requirement already satisfied: humanfriendly in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (4.18)
Requirement already satisfied: alembic in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (1.2.1)
Requirement already satisfied: aiohttp in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (3.6.2)
Requirement already satisfied: python-dateutil<2.8.1,>=2.1 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (2.8.0)
Requirement already satisfied: astor>=0.6.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from tensorflow==1.14.0->ToxicCommentClassification===20200213131414-B4566D) (0.7.1)
Requirement already satisfied: tensorflow-estimator<1.15.0rc0,>=1.14.0rc0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from tensorflow==1.14.0->ToxicCommentClassification===20200213131414-B4566D) (1.14.0)
Requirement already satisfied: wheel>=0.26 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from tensorflow==1.14.0->ToxicCommentClassification===20200213131414-B4566D) (0.33.6)
Requirement already satisfied: gast>=0.2.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from tensorflow==1.14.0->ToxicCommentClassification===20200213131414-B4566D) (0.2.2)
Requirement already satisfied: google-pasta>=0.1.6 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from tensorflow==1.14.0->ToxicCommentClassification===20200213131414-B4566D) (0.1.8)
Requirement already satisfied: tensorboard<1.15.0,>=1.14.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from tensorflow==1.14.0->ToxicCommentClassification===20200213131414-B4566D) (1.14.0)
Requirement already satisfied: termcolor>=1.1.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from tensorflow==1.14.0->ToxicCommentClassification===20200213131414-B4566D) (1.1.0)
Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from tensorflow==1.14.0->ToxicCommentClassification===20200213131414-B4566D) (1.0.9)
Requirement already satisfied: keras-applications>=1.0.6 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from tensorflow==1.14.0->ToxicCommentClassification===20200213131414-B4566D) (1.0.8)
Requirement already satisfied: absl-py>=0.7.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from tensorflow==1.14.0->ToxicCommentClassification===20200213131414-B4566D) (0.7.1)
Requirement already satisfied: wrapt>=1.11.1 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from tensorflow==1.14.0->ToxicCommentClassification===20200213131414-B4566D) (1.11.1)
Requirement already satisfied: six>=1.10.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from tensorflow==1.14.0->ToxicCommentClassification===20200213131414-B4566D) (1.14.0)
Requirement already satisfied: pyyaml in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from keras->ToxicCommentClassification===20200213131414-B4566D) (5.1)
Requirement already satisfied: h5py in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from keras->ToxicCommentClassification===20200213131414-B4566D) (2.9.0)
Requirement already satisfied: scipy>=0.14 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from keras->ToxicCommentClassification===20200213131414-B4566D) (1.2.1)
Requirement already satisfied: pytz>=2011k in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from pandas->ToxicCommentClassification===20200213131414-B4566D) (2019.1)
Requirement already satisfied: ruamel.yaml.clib>=0.1.2; platform_python_implementation == "CPython" and python_version < "3.8" in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from ruamel.yaml>=0.15.0->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (0.2.0)
Requirement already satisfied: Jinja2>=2.10 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from flask->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (2.10.1)
Requirement already satisfied: itsdangerous>=0.24 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from flask->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (1.1.0)
Requirement already satisfied: Werkzeug>=0.14 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from flask->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (0.15.4)
Requirement already satisfied: s3transfer<0.3.0,>=0.2.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from boto3->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (0.2.1)
Requirement already satisfied: botocore<1.14.0,>=1.13.42 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from boto3->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (1.13.42)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from boto3->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (0.9.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from requests->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (1.24.1)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from requests->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (2.8)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from requests->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from requests->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (2019.11.28)
Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from packaging->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (2.4.0)
Requirement already satisfied: websocket-client>=0.32.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from docker->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (0.57.0)
Requirement already satisfied: setuptools in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from protobuf>=3.6.0->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (45.1.0.post20200119)
Requirement already satisfied: python-editor>=0.3 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from alembic->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (1.0.4)
Requirement already satisfied: Mako in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from alembic->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (1.1.0)
Requirement already satisfied: multidict<5.0,>=4.5 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from aiohttp->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (4.7.4)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from aiohttp->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (1.4.2)
Requirement already satisfied: async-timeout<4.0,>=3.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from aiohttp->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (3.0.1)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from aiohttp->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (19.1.0)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from tensorboard<1.15.0,>=1.14.0->tensorflow==1.14.0->ToxicCommentClassification===20200213131414-B4566D) (3.1)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from Jinja2>=2.10->flask->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (1.1.1)
Requirement already satisfied: docutils<0.16,>=0.10 in /usr/local/anaconda3/envs/dev-py3/lib/python3.7/site-packages (from botocore<1.14.0,>=1.13.42->boto3->bentoml==0.6.2->ToxicCommentClassification===20200213131414-B4566D) (0.15.2)
Building wheels for collected packages: ToxicCommentClassification
  Building wheel for ToxicCommentClassification (setup.py) ... done
  Created wheel for ToxicCommentClassification: filename=ToxicCommentClassification-20200213131414_B4566D-py3-none-any.whl size=16121004 sha256=69af4300684b8598686bcf4707c5a5044d52425ac2e671ed1542981c2741312c
  Stored in directory: /private/var/folders/kn/xnc9k74x03567n1mx2tfqnpr0000gn/T/pip-ephem-wheel-cache-8ycwo0oi/wheels/ef/00/db/6c76f4e5d4f074deccf533c31c3bb255c8f5cb43809457dffd
Successfully built ToxicCommentClassification
Installing collected packages: ToxicCommentClassification
Successfully installed ToxicCommentClassification-20200213131414-B4566D
In [30]:
import ToxicCommentClassification

svc = ToxicCommentClassification.load()
result = svc.predict(sample_test)
result
Out[30]:
['toxic', 'toxic']

Deploy BentoService as REST API server to the cloud

BentoML support deployment to multiply cloud provider services, such as AWS Lambda, AWS Sagemaker, Google Cloudrun and etc. You can find the full list and guide on the documentation site at https://docs.bentoml.org/en/latest/deployment/index.html

For this project, we are going to deploy to AWS Sagemaker

Use bentoml sagemaker deploy to deploy BentoService to AWS Sagemaker

In [28]:
!bentoml sagemaker deploy keras-toxic -b ToxicCommentClassification:20200213141504_137C94 \
    --api-name predict --verbose
[2020-02-13 14:17:10,940] DEBUG - Using BentoML with local Yatai server
[2020-02-13 14:17:11,041] DEBUG - Upgrading tables to the latest revision
Deploying Sagemaker deployment \[2020-02-13 14:17:11,841] DEBUG - Created temporary directory: /private/var/folders/kn/xnc9k74x03567n1mx2tfqnpr0000gn/T/bentoml-temp-mblyir49
\[2020-02-13 14:17:12,274] DEBUG - Getting docker login info from AWS
[2020-02-13 14:17:12,275] DEBUG - Building docker image: 192023623294.dkr.ecr.us-west-2.amazonaws.com/toxiccommentclassification-sagemaker:20200213141504_137C94
-[2020-02-13 14:17:13,496] INFO - Step 1/11 : FROM continuumio/miniconda3:4.7.12
[2020-02-13 14:17:13,497] INFO - 

[2020-02-13 14:17:13,497] INFO -  ---> 406f2b43ea59

[2020-02-13 14:17:13,497] INFO - Step 2/11 : EXPOSE 8080
[2020-02-13 14:17:13,498] INFO - 

[2020-02-13 14:17:13,498] INFO -  ---> Using cache

[2020-02-13 14:17:13,498] INFO -  ---> 85b4bb5fff81

[2020-02-13 14:17:13,498] INFO - Step 3/11 : RUN set -x      && apt-get update      && apt-get install --no-install-recommends --no-install-suggests -y libpq-dev build-essential     && apt-get install -y nginx      && rm -rf /var/lib/apt/lists/*
[2020-02-13 14:17:13,498] INFO - 

[2020-02-13 14:17:13,498] INFO -  ---> Using cache

[2020-02-13 14:17:13,499] INFO -  ---> 66280bc479d1

[2020-02-13 14:17:13,499] INFO - Step 4/11 : RUN conda install pip numpy scipy       && pip install gunicorn gevent
[2020-02-13 14:17:13,499] INFO - 

[2020-02-13 14:17:13,499] INFO -  ---> Using cache

[2020-02-13 14:17:13,499] INFO -  ---> 1dbfc408f11d

[2020-02-13 14:17:13,499] INFO - Step 5/11 : COPY . /opt/program
[2020-02-13 14:17:13,499] INFO - 

|[2020-02-13 14:17:14,140] INFO -  ---> cba7cba26390

[2020-02-13 14:17:14,141] INFO - Step 6/11 : WORKDIR /opt/program
[2020-02-13 14:17:14,141] INFO - 

\[2020-02-13 14:17:14,235] INFO -  ---> Running in 194a1db36a87

|[2020-02-13 14:17:14,541] INFO -  ---> 0da0c5d76587

[2020-02-13 14:17:14,541] INFO - Step 7/11 : RUN conda env update -n base -f /opt/program/environment.yml
[2020-02-13 14:17:14,541] INFO - 

\[2020-02-13 14:17:14,655] INFO -  ---> Running in 080847045f66

/[2020-02-13 14:17:16,139] INFO - Collecting package metadata (repodata.json): 
[2020-02-13 14:17:16,140] INFO - ...working... 
-[2020-02-13 14:17:21,679] INFO - done
Solving environment: ...working... 
\[2020-02-13 14:17:27,704] INFO - done

-[2020-02-13 14:17:27,801] INFO - 
Downloading and Extracting Packages
python-3.7.3         | 32.1 MB   |            |   0% 
python-3.7.3         | 32.1 MB   |            |   0% 
python-3.7.3         | 32.1 MB   |            |   1% 
python-3.7.3         | 32.1 MB   | 2          |   2% 
python-3.7.3         | 32.1 MB   | 5          |   6% 
python-3.7.3         | 32.1 MB   | 8          |   8% 
python-3.7.3         | 32.1 MB   | #          |  10% 
python-3.7.3         | 32.1 MB   | #2         |  12% 
python-3.7.3         | 32.1 MB   | #3         |  14% 
python-3.7.3         | 32.1 MB   | #5         |  15% 
python-3.7.3         | 32.1 MB   | #7         |  17% 
python-3.7.3         | 32.1 MB   | #8         |  19% 
python-3.7.3         | 32.1 MB   | ##1        |  21% 
python-3.7.3         | 32.1 MB   | ##2        |  23% 
python-3.7.3         | 32.1 MB   | ##4        |  24% 
python-3.7.3         | 32.1 MB   | ##6        |  26% 
python-3.7.3         | 32.1 MB   | ##8        |  28% 
python-3.7.3         | 32.1 MB   | ###        |  30% 
python-3.7.3         | 32.1 MB   | ###2       |  32% 
python-3.7.3         | 32.1 MB   | ###4       |  34% 
python-3.7.3         | 32.1 MB   | ###6       |  36% 
python-3.7.3         | 32.1 MB   | ###7       |  38% 
python-3.7.3         | 32.1 MB   | ####       |  40% 
python-3.7.3         | 32.1 MB   | ####2      |  43% 
python-3.7.3         | 32.1 MB   | ####4      |  45% 
python-3.7.3         | 32.1 MB   | ####6      |  47% 
python-3.7.3         | 32.1 MB   | ####8      |  49% 
python-3.7.3         | 32.1 MB   | #####1     |  52% 
python-3.7.3         | 32.1 MB   | #####3     |  54% 
python-3.7.3         | 32.1 MB   | #####5     |  56% 
python-3.7.3         | 32.1 MB   | #####7     |  58% 
python-3.7.3         | 32.1 MB   | ######1    |  61% 
python-3.7.3         | 32.1 MB   | ######3    |  63% 
python-3.7.3         | 32.1 MB   | ######5    |  65% 
python-3.7.3         | 32.1 MB   | ######7    |  68% 
python-3.7.3         | 32.1 MB   | #######    |  71% 
python-3.7.3         | 32.1 MB   | #######3   |  73% 
python-3.7.3         | 32.1 MB   | #######6   |  77% 
python-3.7.3         | 32.1 MB   | #######9   |  79% 
python-3.7.3         | 32.1 MB   | ########1  |  82% 
python-3.7.3         | 32.1 MB   | ########4  |  84% 
python-3.7.3         | 32.1 MB   | ########7  |  88% 
python-3.7.3         | 32.1 MB   | ########9  |  90% 
python-3.7.3         | 32.1 MB   | #########2 |  92% 
python-3.7.3         | 32.1 MB   | #########5 |  95% 
python-3.7.3         | 32.1 MB   | #########8 |  99% 
python-3.7.3         | 32.1 MB   | ########## | 100% 
[2020-02-13 14:17:34,776] INFO - 
Preparing transaction: 
[2020-02-13 14:17:34,776] INFO - ...working... 
/[2020-02-13 14:17:34,904] INFO - done

[2020-02-13 14:17:34,904] INFO - Verifying transaction: ...working... 
\[2020-02-13 14:17:35,460] INFO - done

[2020-02-13 14:17:35,460] INFO - Executing transaction: ...working... 
|[2020-02-13 14:17:39,440] INFO - done

\[2020-02-13 14:17:39,992] INFO - #
# To activate this environment, use
#
#     $ conda activate base
#
# To deactivate an active environment, use
#
#     $ conda deactivate


/[2020-02-13 14:17:43,856] INFO -  ---> 54bd606efbc7

|[2020-02-13 14:17:43,858] INFO - Step 8/11 : RUN pip install -r /opt/program/requirements.txt
[2020-02-13 14:17:43,859] INFO - 

\[2020-02-13 14:17:44,021] INFO -  ---> Running in 403ba77bca08

\[2020-02-13 14:17:45,613] INFO - Collecting bentoml==0.6.2

-[2020-02-13 14:17:45,692] INFO -   Downloading BentoML-0.6.2-py3-none-any.whl (554 kB)

|[2020-02-13 14:17:46,365] INFO - Collecting tensorflow==1.14.0

[2020-02-13 14:17:46,375] INFO -   Downloading tensorflow-1.14.0-cp37-cp37m-manylinux1_x86_64.whl (109.3 MB)

\[2020-02-13 14:18:19,837] INFO - Collecting keras

[2020-02-13 14:18:19,848] INFO -   Downloading Keras-2.3.1-py2.py3-none-any.whl (377 kB)

-[2020-02-13 14:18:20,373] INFO - Collecting pandas

[2020-02-13 14:18:20,385] INFO -   Downloading pandas-1.0.1-cp37-cp37m-manylinux1_x86_64.whl (10.1 MB)

-[2020-02-13 14:18:23,180] INFO - Requirement already satisfied: numpy in /opt/conda/lib/python3.7/site-packages (from -r /opt/program/requirements.txt (line 5)) (1.18.1)

[2020-02-13 14:18:23,248] INFO - Collecting packaging

/[2020-02-13 14:18:23,264] INFO -   Downloading packaging-20.1-py2.py3-none-any.whl (36 kB)

|[2020-02-13 14:18:23,371] INFO - Collecting docker

[2020-02-13 14:18:23,382] INFO -   Downloading docker-4.2.0-py2.py3-none-any.whl (143 kB)

[2020-02-13 14:18:23,450] INFO - Requirement already satisfied: gunicorn in /opt/conda/lib/python3.7/site-packages (from bentoml==0.6.2->-r /opt/program/requirements.txt (line 1)) (20.0.4)

-[2020-02-13 14:18:24,444] INFO - Collecting grpcio

[2020-02-13 14:18:24,454] INFO -   Downloading grpcio-1.27.1-cp37-cp37m-manylinux2010_x86_64.whl (2.7 MB)

-[2020-02-13 14:18:25,274] INFO - Collecting alembic

/[2020-02-13 14:18:25,306] INFO -   Downloading alembic-1.4.0.tar.gz (1.1 MB)

-[2020-02-13 14:18:26,058] INFO - Collecting python-dateutil<2.8.1,>=2.1

[2020-02-13 14:18:26,069] INFO -   Downloading python_dateutil-2.8.0-py2.py3-none-any.whl (226 kB)

/[2020-02-13 14:18:26,178] INFO - Collecting click>=7.0

[2020-02-13 14:18:26,189] INFO -   Downloading Click-7.0-py2.py3-none-any.whl (81 kB)

|[2020-02-13 14:18:27,141] INFO - Collecting ruamel.yaml>=0.15.0

\[2020-02-13 14:18:27,153] INFO -   Downloading ruamel.yaml-0.16.10-py2.py3-none-any.whl (111 kB)

-[2020-02-13 14:18:27,327] INFO - Collecting humanfriendly

[2020-02-13 14:18:27,340] INFO -   Downloading humanfriendly-6.1-py2.py3-none-any.whl (75 kB)

/[2020-02-13 14:18:27,423] INFO - Collecting prometheus-client

[2020-02-13 14:18:27,435] INFO -   Downloading prometheus_client-0.7.1.tar.gz (38 kB)

/[2020-02-13 14:18:27,828] INFO - Collecting sqlalchemy>=1.3.0

[2020-02-13 14:18:27,839] INFO -   Downloading SQLAlchemy-1.3.13.tar.gz (6.0 MB)

\[2020-02-13 14:18:30,884] INFO - Collecting cerberus

[2020-02-13 14:18:30,900] INFO -   Downloading Cerberus-1.3.2.tar.gz (52 kB)

\[2020-02-13 14:18:31,249] INFO - Collecting python-json-logger

[2020-02-13 14:18:31,265] INFO -   Downloading python-json-logger-0.1.11.tar.gz (6.0 kB)

|[2020-02-13 14:18:31,579] INFO - Collecting tabulate

[2020-02-13 14:18:31,590] INFO -   Downloading tabulate-0.8.6.tar.gz (45 kB)

|[2020-02-13 14:18:31,941] INFO - Collecting configparser

[2020-02-13 14:18:31,953] INFO -   Downloading configparser-4.0.2-py2.py3-none-any.whl (22 kB)

-[2020-02-13 14:18:32,168] INFO - Collecting protobuf>=3.6.0

[2020-02-13 14:18:32,181] INFO -   Downloading protobuf-3.11.3-cp37-cp37m-manylinux1_x86_64.whl (1.3 MB)

\[2020-02-13 14:18:32,483] INFO - Requirement already satisfied: requests in /opt/conda/lib/python3.7/site-packages (from bentoml==0.6.2->-r /opt/program/requirements.txt (line 1)) (2.22.0)

-[2020-02-13 14:18:32,555] INFO - Collecting flask

[2020-02-13 14:18:32,565] INFO -   Downloading Flask-1.1.1-py2.py3-none-any.whl (94 kB)

\[2020-02-13 14:18:33,334] INFO - Collecting boto3

[2020-02-13 14:18:33,348] INFO -   Downloading boto3-1.11.17-py2.py3-none-any.whl (128 kB)

-[2020-02-13 14:18:33,439] INFO - Collecting google-pasta>=0.1.6

[2020-02-13 14:18:33,454] INFO -   Downloading google_pasta-0.1.8-py3-none-any.whl (57 kB)

/[2020-02-13 14:18:33,519] INFO - Collecting keras-applications>=1.0.6

[2020-02-13 14:18:33,533] INFO -   Downloading Keras_Applications-1.0.8-py3-none-any.whl (50 kB)

|[2020-02-13 14:18:33,604] INFO - Collecting absl-py>=0.7.0

[2020-02-13 14:18:33,626] INFO -   Downloading absl-py-0.9.0.tar.gz (104 kB)

/[2020-02-13 14:18:33,917] INFO - Requirement already satisfied: wheel>=0.26 in /opt/conda/lib/python3.7/site-packages (from tensorflow==1.14.0->-r /opt/program/requirements.txt (line 2)) (0.33.6)

[2020-02-13 14:18:33,964] INFO - Collecting keras-preprocessing>=1.0.5

[2020-02-13 14:18:33,978] INFO -   Downloading Keras_Preprocessing-1.1.0-py2.py3-none-any.whl (41 kB)

|[2020-02-13 14:18:34,050] INFO - Collecting wrapt>=1.11.1

[2020-02-13 14:18:34,064] INFO -   Downloading wrapt-1.11.2.tar.gz (27 kB)

|[2020-02-13 14:18:34,422] INFO - Collecting gast>=0.2.0

[2020-02-13 14:18:34,433] INFO -   Downloading gast-0.3.3-py2.py3-none-any.whl (9.7 kB)

[2020-02-13 14:18:34,455] INFO - Requirement already satisfied: six>=1.10.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow==1.14.0->-r /opt/program/requirements.txt (line 2)) (1.12.0)

[2020-02-13 14:18:34,485] INFO - Collecting termcolor>=1.1.0

[2020-02-13 14:18:34,495] INFO -   Downloading termcolor-1.1.0.tar.gz (3.9 kB)

/[2020-02-13 14:18:34,806] INFO - Collecting tensorflow-estimator<1.15.0rc0,>=1.14.0rc0

|[2020-02-13 14:18:34,819] INFO -   Downloading tensorflow_estimator-1.14.0-py2.py3-none-any.whl (488 kB)

-[2020-02-13 14:18:35,049] INFO - Collecting astor>=0.6.0

[2020-02-13 14:18:35,063] INFO -   Downloading astor-0.8.1-py2.py3-none-any.whl (27 kB)

/[2020-02-13 14:18:35,168] INFO - Collecting tensorboard<1.15.0,>=1.14.0

[2020-02-13 14:18:35,186] INFO -   Downloading tensorboard-1.14.0-py3-none-any.whl (3.1 MB)

|[2020-02-13 14:18:36,086] INFO - Collecting pyyaml

[2020-02-13 14:18:36,105] INFO -   Downloading PyYAML-5.3.tar.gz (268 kB)

\[2020-02-13 14:18:36,621] INFO - Requirement already satisfied: scipy>=0.14 in /opt/conda/lib/python3.7/site-packages (from keras->-r /opt/program/requirements.txt (line 3)) (1.4.1)

/[2020-02-13 14:18:36,749] INFO - Collecting h5py

[2020-02-13 14:18:36,760] INFO -   Downloading h5py-2.10.0-cp37-cp37m-manylinux1_x86_64.whl (2.9 MB)

|[2020-02-13 14:18:37,709] INFO - Collecting pytz>=2017.2

[2020-02-13 14:18:37,722] INFO -   Downloading pytz-2019.3-py2.py3-none-any.whl (509 kB)

|[2020-02-13 14:18:38,099] INFO - Collecting pyparsing>=2.0.2

[2020-02-13 14:18:38,112] INFO -   Downloading pyparsing-2.4.6-py2.py3-none-any.whl (67 kB)

\[2020-02-13 14:18:38,215] INFO - Collecting websocket-client>=0.32.0

[2020-02-13 14:18:38,239] INFO -   Downloading websocket_client-0.57.0-py2.py3-none-any.whl (200 kB)

-[2020-02-13 14:18:38,306] INFO - Requirement already satisfied: setuptools>=3.0 in /opt/conda/lib/python3.7/site-packages (from gunicorn->bentoml==0.6.2->-r /opt/program/requirements.txt (line 1)) (41.4.0)

[2020-02-13 14:18:38,361] INFO - Collecting Mako

[2020-02-13 14:18:38,379] INFO -   Downloading Mako-1.1.1.tar.gz (468 kB)

|[2020-02-13 14:18:38,950] INFO - Collecting python-editor>=0.3

[2020-02-13 14:18:38,963] INFO -   Downloading python_editor-1.0.4-py3-none-any.whl (4.9 kB)

\[2020-02-13 14:18:39,099] INFO - Collecting ruamel.yaml.clib>=0.1.2; platform_python_implementation == "CPython" and python_version < "3.9"

-[2020-02-13 14:18:39,114] INFO -   Downloading ruamel.yaml.clib-0.2.0-cp37-cp37m-manylinux1_x86_64.whl (547 kB)

/[2020-02-13 14:18:39,270] INFO - Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.6.2->-r /opt/program/requirements.txt (line 1)) (2.8)

[2020-02-13 14:18:39,273] INFO - Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.6.2->-r /opt/program/requirements.txt (line 1)) (2019.11.28)

[2020-02-13 14:18:39,275] INFO - Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.6.2->-r /opt/program/requirements.txt (line 1)) (3.0.4)

[2020-02-13 14:18:39,278] INFO - Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.6.2->-r /opt/program/requirements.txt (line 1)) (1.24.2)

|[2020-02-13 14:18:39,323] INFO - Collecting itsdangerous>=0.24

[2020-02-13 14:18:39,338] INFO -   Downloading itsdangerous-1.1.0-py2.py3-none-any.whl (16 kB)

\[2020-02-13 14:18:39,441] INFO - Collecting Werkzeug>=0.15

[2020-02-13 14:18:39,453] INFO -   Downloading Werkzeug-1.0.0-py2.py3-none-any.whl (298 kB)

/[2020-02-13 14:18:39,628] INFO - Collecting Jinja2>=2.10.1

[2020-02-13 14:18:39,641] INFO -   Downloading Jinja2-2.11.1-py2.py3-none-any.whl (126 kB)

|[2020-02-13 14:18:39,736] INFO - Collecting jmespath<1.0.0,>=0.7.1

[2020-02-13 14:18:39,749] INFO -   Downloading jmespath-0.9.4-py2.py3-none-any.whl (24 kB)

|[2020-02-13 14:18:40,548] INFO - Collecting botocore<1.15.0,>=1.14.17

[2020-02-13 14:18:40,559] INFO -   Downloading botocore-1.14.17-py2.py3-none-any.whl (5.9 MB)

/[2020-02-13 14:18:42,532] INFO - Collecting s3transfer<0.4.0,>=0.3.0

[2020-02-13 14:18:42,551] INFO -   Downloading s3transfer-0.3.3-py2.py3-none-any.whl (69 kB)

-[2020-02-13 14:18:42,876] INFO - Collecting markdown>=2.6.8

/[2020-02-13 14:18:42,888] INFO -   Downloading Markdown-3.2.1-py2.py3-none-any.whl (88 kB)

|[2020-02-13 14:18:42,993] INFO - Collecting MarkupSafe>=0.9.2

[2020-02-13 14:18:43,006] INFO -   Downloading MarkupSafe-1.1.1-cp37-cp37m-manylinux1_x86_64.whl (27 kB)

[2020-02-13 14:18:43,075] INFO - Collecting docutils<0.16,>=0.10

\[2020-02-13 14:18:43,090] INFO -   Downloading docutils-0.15.2-py3-none-any.whl (547 kB)

-[2020-02-13 14:18:43,260] INFO - Building wheels for collected packages: alembic, prometheus-client, sqlalchemy, cerberus, python-json-logger, tabulate, absl-py, wrapt, termcolor, pyyaml, Mako

[2020-02-13 14:18:43,261] INFO -   Building wheel for alembic (setup.py): started

/[2020-02-13 14:18:43,764] INFO -   Building wheel for alembic (setup.py): finished with status 'done'

[2020-02-13 14:18:43,766] INFO -   Created wheel for alembic: filename=alembic-1.4.0-py2.py3-none-any.whl size=157563 sha256=357bbb2ccd9e93f70d3359d3f195b83cafbd43ae4ae0c852febca0c96582454e

[2020-02-13 14:18:43,766] INFO -   Stored in directory: /root/.cache/pip/wheels/33/a9/f9/a53f885636269db5b76cf7afa3a1ab86d9d2fe96610d09274e

[2020-02-13 14:18:43,769] INFO -   Building wheel for prometheus-client (setup.py): started

/[2020-02-13 14:18:44,123] INFO -   Building wheel for prometheus-client (setup.py): finished with status 'done'

[2020-02-13 14:18:44,124] INFO -   Created wheel for prometheus-client: filename=prometheus_client-0.7.1-py3-none-any.whl size=41402 sha256=3363ccf54ac9bb5b90f55f6407bd2237a3d577bd11bad36df150adfcf2b36b35
  Stored in directory: /root/.cache/pip/wheels/30/0c/26/59ba285bf65dc79d195e9b25e2ddde4c61070422729b0cd914

[2020-02-13 14:18:44,127] INFO -   Building wheel for sqlalchemy (setup.py): started

-[2020-02-13 14:18:46,103] INFO -   Building wheel for sqlalchemy (setup.py): finished with status 'done'

[2020-02-13 14:18:46,108] INFO -   Created wheel for sqlalchemy: filename=SQLAlchemy-1.3.13-cp37-cp37m-linux_x86_64.whl size=1223705 sha256=96f2931d8d4aab0357a906f7d2886db52ef463f7d5c658db467aac5148db398f
  Stored in directory: /root/.cache/pip/wheels/b9/ba/77/163f10f14bd489351530603e750c195b0ceceed2f3be2b32f1

[2020-02-13 14:18:46,110] INFO -   Building wheel for cerberus (setup.py): started

\[2020-02-13 14:18:46,440] INFO -   Building wheel for cerberus (setup.py): finished with status 'done'

[2020-02-13 14:18:46,441] INFO -   Created wheel for cerberus: filename=Cerberus-1.3.2-py3-none-any.whl size=54335 sha256=950cf9bdaff98c4c9c442537d2f077af382916bc35e0556860dc2160e2740999

[2020-02-13 14:18:46,442] INFO -   Stored in directory: /root/.cache/pip/wheels/17/3a/0d/e2fc48cf85cb858f5e65f1baa36180ebb5dce6397c35c4cfcb

[2020-02-13 14:18:46,444] INFO -   Building wheel for python-json-logger (setup.py): started

|[2020-02-13 14:18:46,767] INFO -   Building wheel for python-json-logger (setup.py): finished with status 'done'

[2020-02-13 14:18:46,767] INFO -   Created wheel for python-json-logger: filename=python_json_logger-0.1.11-py2.py3-none-any.whl size=5076 sha256=cfadb6fbf864a860b3a483bbd66516d5c85ec0770a43b20c5a919c3d25ec0b41

[2020-02-13 14:18:46,767] INFO -   Stored in directory: /root/.cache/pip/wheels/fa/7f/fd/92ccdbb9d1a65486406e0363d2ba5b4ce52f400a915f602ecb

[2020-02-13 14:18:46,769] INFO -   Building wheel for tabulate (setup.py): started

/[2020-02-13 14:18:47,072] INFO -   Building wheel for tabulate (setup.py): finished with status 'done'

[2020-02-13 14:18:47,073] INFO -   Created wheel for tabulate: filename=tabulate-0.8.6-py3-none-any.whl size=23273 sha256=20a728bcd4d63e632d1f15ffb80ed5d308f160db6c3f7d1bdf2c9eb961af60ce
  Stored in directory: /root/.cache/pip/wheels/09/b6/7e/08b4ee715a1239453e89a59081f0ac369a9036f232e013ecd8

[2020-02-13 14:18:47,075] INFO -   Building wheel for absl-py (setup.py): started

/[2020-02-13 14:18:47,406] INFO -   Building wheel for absl-py (setup.py): finished with status 'done'

[2020-02-13 14:18:47,407] INFO -   Created wheel for absl-py: filename=absl_py-0.9.0-py3-none-any.whl size=121931 sha256=627095a53ed0678f2136caa7bd95ac2a1137d322b9a2aba0a064b089eae0ccad

[2020-02-13 14:18:47,407] INFO -   Stored in directory: /root/.cache/pip/wheels/cc/af/1a/498a24d0730ef484019e007bb9e8cef3ac00311a672c049a3e

[2020-02-13 14:18:47,409] INFO -   Building wheel for wrapt (setup.py): started

\[2020-02-13 14:18:48,490] INFO -   Building wheel for wrapt (setup.py): finished with status 'done'

[2020-02-13 14:18:48,491] INFO -   Created wheel for wrapt: filename=wrapt-1.11.2-cp37-cp37m-linux_x86_64.whl size=76131 sha256=d2110e709b73f05f0c4493db741fbce3fbf320e8abde5b4f3f073bf9f130c1a4
  Stored in directory: /root/.cache/pip/wheels/23/5f/62/304b411f20be41821465a82bc98baabc5e68c3cdd1eb99db71

[2020-02-13 14:18:48,493] INFO -   Building wheel for termcolor (setup.py): started

|[2020-02-13 14:18:48,792] INFO -   Building wheel for termcolor (setup.py): finished with status 'done'

[2020-02-13 14:18:48,793] INFO -   Created wheel for termcolor: filename=termcolor-1.1.0-py3-none-any.whl size=4830 sha256=9eac67181e3c1bd4a86b64f979eaaa69706dff92e78133943d2195d240b5bcf1

[2020-02-13 14:18:48,793] INFO -   Stored in directory: /root/.cache/pip/wheels/3f/e3/ec/8a8336ff196023622fbcb36de0c5a5c218cbb24111d1d4c7f2

[2020-02-13 14:18:48,795] INFO -   Building wheel for pyyaml (setup.py): started

\[2020-02-13 14:18:49,283] INFO -   Building wheel for pyyaml (setup.py): finished with status 'done'

[2020-02-13 14:18:49,284] INFO -   Created wheel for pyyaml: filename=PyYAML-5.3-cp37-cp37m-linux_x86_64.whl size=44228 sha256=89d082d491b88b59beb31d97fb56415821249635e14170edc5e31b7865780473
  Stored in directory: /root/.cache/pip/wheels/8a/55/a4/c0a81d27c33462cfdcb904db018f5550197e88b2b6b85beed2

[2020-02-13 14:18:49,286] INFO -   Building wheel for Mako (setup.py): started

\[2020-02-13 14:18:49,684] INFO -   Building wheel for Mako (setup.py): finished with status 'done'

[2020-02-13 14:18:49,685] INFO -   Created wheel for Mako: filename=Mako-1.1.1-py3-none-any.whl size=75409 sha256=d7d9d2cb41ba1b5ed5958abe60e550aa79b8db04a7715e53640c16d4ecd1c28d

[2020-02-13 14:18:49,685] INFO -   Stored in directory: /root/.cache/pip/wheels/11/fe/fa/3693b62cf5ec2b2784b6496734f0ee3e2321eb66d66607e5f9

[2020-02-13 14:18:49,686] INFO - Successfully built alembic prometheus-client sqlalchemy cerberus python-json-logger tabulate absl-py wrapt termcolor pyyaml Mako

/[2020-02-13 14:18:50,293] INFO - Installing collected packages: pyparsing, packaging, websocket-client, docker, grpcio, sqlalchemy, MarkupSafe, Mako, python-editor, python-dateutil, alembic, click, ruamel.yaml.clib, ruamel.yaml, humanfriendly, pytz, pandas, prometheus-client, cerberus, python-json-logger, tabulate, configparser, protobuf, itsdangerous, Werkzeug, Jinja2, flask, jmespath, docutils, botocore, s3transfer, boto3, bentoml, google-pasta, h5py, keras-applications, absl-py, keras-preprocessing, wrapt, gast, termcolor, tensorflow-estimator, astor, markdown, tensorboard, tensorflow, pyyaml, keras

\[2020-02-13 14:19:20,620] INFO - Successfully installed Jinja2-2.11.1 Mako-1.1.1 MarkupSafe-1.1.1 Werkzeug-1.0.0 absl-py-0.9.0 alembic-1.4.0 astor-0.8.1 bentoml-0.6.2 boto3-1.11.17 botocore-1.14.17 cerberus-1.3.2 click-7.0 configparser-4.0.2 docker-4.2.0 docutils-0.15.2 flask-1.1.1 gast-0.3.3 google-pasta-0.1.8 grpcio-1.27.1 h5py-2.10.0 humanfriendly-6.1 itsdangerous-1.1.0 jmespath-0.9.4 keras-2.3.1 keras-applications-1.0.8 keras-preprocessing-1.1.0 markdown-3.2.1 packaging-20.1 pandas-1.0.1 prometheus-client-0.7.1 protobuf-3.11.3 pyparsing-2.4.6 python-dateutil-2.8.0 python-editor-1.0.4 python-json-logger-0.1.11 pytz-2019.3 pyyaml-5.3 ruamel.yaml-0.16.10 ruamel.yaml.clib-0.2.0 s3transfer-0.3.3 sqlalchemy-1.3.13 tabulate-0.8.6 tensorboard-1.14.0 tensorflow-1.14.0 tensorflow-estimator-1.14.0 termcolor-1.1.0 websocket-client-0.57.0 wrapt-1.11.2

|[2020-02-13 14:19:39,302] INFO -  ---> dd27146f42c8

[2020-02-13 14:19:39,304] INFO - Step 9/11 : RUN if [ -f /bento/bentoml_init.sh ]; then /bin/bash -c /bento/bentoml_init.sh; fi
[2020-02-13 14:19:39,305] INFO - 

-[2020-02-13 14:19:39,448] INFO -  ---> Running in d7b2f9bebbf7

/[2020-02-13 14:19:40,801] INFO -  ---> 8d0862e3b1d9

[2020-02-13 14:19:40,801] INFO - Step 10/11 : RUN if [ -f /opt/program/setup.sh ]; then /bin/bash -c /opt/program/setup.sh; fi
[2020-02-13 14:19:40,801] INFO - 

\[2020-02-13 14:19:40,984] INFO -  ---> Running in 989e45fda267

/[2020-02-13 14:19:42,440] INFO -  ---> 69f553dc4083

[2020-02-13 14:19:42,440] INFO - Step 11/11 : ENV PATH="/opt/program:${PATH}"
[2020-02-13 14:19:42,441] INFO - 

\[2020-02-13 14:19:42,563] INFO -  ---> Running in 7cda9734f76d

/[2020-02-13 14:19:42,809] INFO -  ---> b8d3545c59be

[2020-02-13 14:19:42,811] INFO - Successfully built b8d3545c59be

[2020-02-13 14:19:42,815] INFO - Successfully tagged 192023623294.dkr.ecr.us-west-2.amazonaws.com/toxiccommentclassification-sagemaker:20200213141504_137C94

\[2020-02-13 14:19:43,065] DEBUG - Pushing image to AWS ECR at 192023623294.dkr.ecr.us-west-2.amazonaws.com/toxiccommentclassification-sagemaker:20200213141504_137C94
/[2020-02-13 14:21:11,668] DEBUG - Finished pushing image: 192023623294.dkr.ecr.us-west-2.amazonaws.com/toxiccommentclassification-sagemaker:20200213141504_137C94
[2020-02-13 14:21:11,674] DEBUG - BentoML in debug mode, keeping temp directory "/private/var/folders/kn/xnc9k74x03567n1mx2tfqnpr0000gn/T/bentoml-temp-mblyir49"
\[2020-02-13 14:21:13,122] DEBUG - AWS create model response: {'ModelArn': 'arn:aws:sagemaker:us-west-2:192023623294:model/bobo-keras-toxic-toxiccommentclassifi-20200213141504-137c94', 'ResponseMetadata': {'RequestId': 'c3254d61-99fd-4b0f-9a84-9f4b3b8c1542', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'c3254d61-99fd-4b0f-9a84-9f4b3b8c1542', 'content-type': 'application/x-amz-json-1.1', 'content-length': '121', 'date': 'Thu, 13 Feb 2020 22:21:12 GMT'}, 'RetryAttempts': 0}}
[2020-02-13 14:21:13,122] DEBUG - Creating Sagemaker endpoint bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94 configuration
-[2020-02-13 14:21:13,199] DEBUG - AWS create endpoint config response: {'EndpointConfigArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint-config/bobo-keras-toxic-toxiccommentclassifi-20200213141504-137c94', 'ResponseMetadata': {'RequestId': 'd9c178b6-cb46-47ac-9f02-f87d0686f860', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'd9c178b6-cb46-47ac-9f02-f87d0686f860', 'content-type': 'application/x-amz-json-1.1', 'content-length': '140', 'date': 'Thu, 13 Feb 2020 22:21:12 GMT'}, 'RetryAttempts': 0}}
[2020-02-13 14:21:13,200] DEBUG - Creating sagemaker endpoint bobo-keras-toxic
|[2020-02-13 14:21:13,401] DEBUG - AWS create endpoint response: {'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'ResponseMetadata': {'RequestId': '5b68def6-2df7-4515-8a22-3efa2a0cf20e', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '5b68def6-2df7-4515-8a22-3efa2a0cf20e', 'content-type': 'application/x-amz-json-1.1', 'content-length': '84', 'date': 'Thu, 13 Feb 2020 22:21:13 GMT'}, 'RetryAttempts': 0}}
[2020-02-13 14:21:13,416] INFO - ApplyDeployment (keras-toxic, namespace bobo) succeeded
\[2020-02-13 14:21:13,593] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '2c3be40c-4560-4f26-a20f-bbbe4e3829c2', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '2c3be40c-4560-4f26-a20f-bbbe4e3829c2', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:21:13 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:21:18,801] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'f9042b2b-a8ba-4375-926b-6500f1afcd55', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'f9042b2b-a8ba-4375-926b-6500f1afcd55', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:21:18 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:21:23,996] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '636ba9ee-b5ee-4ab9-83f4-f5179e9cc636', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '636ba9ee-b5ee-4ab9-83f4-f5179e9cc636', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:21:23 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:21:29,186] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '65bc7b17-bcd9-46e0-b9d9-98a1aaaba140', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '65bc7b17-bcd9-46e0-b9d9-98a1aaaba140', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:21:29 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:21:34,376] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'b8f1a64b-692a-4887-af08-f1301a9c21b9', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'b8f1a64b-692a-4887-af08-f1301a9c21b9', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:21:33 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:21:39,545] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '9ddd7931-562c-45b9-b91b-5d5e7418824a', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '9ddd7931-562c-45b9-b91b-5d5e7418824a', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:21:38 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:21:44,728] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'bd718259-a6b3-4083-a06b-3f9ad81e2e25', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'bd718259-a6b3-4083-a06b-3f9ad81e2e25', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:21:43 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:21:49,922] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '057c4d8d-b429-43f7-ae21-6b6e81289b54', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '057c4d8d-b429-43f7-ae21-6b6e81289b54', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:21:49 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:21:55,115] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '213d0f83-a788-42b7-8ce9-a3fb5d2fbe1a', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '213d0f83-a788-42b7-8ce9-a3fb5d2fbe1a', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:21:54 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:22:00,320] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '5b57327b-b2ce-4ca8-b4cd-c0ca0277bc84', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '5b57327b-b2ce-4ca8-b4cd-c0ca0277bc84', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:21:59 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:22:05,668] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '52611fbb-3ae9-4037-a296-bcb7a5fcfc6e', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '52611fbb-3ae9-4037-a296-bcb7a5fcfc6e', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:22:04 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:22:10,933] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '1b2f2f52-5afa-439a-9a1e-d860a2d5dfba', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '1b2f2f52-5afa-439a-9a1e-d860a2d5dfba', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:22:10 GMT'}, 'RetryAttempts': 0}}
|[2020-02-13 14:22:16,134] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '224e01c0-436a-4f92-8701-5fbe81666ada', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '224e01c0-436a-4f92-8701-5fbe81666ada', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:22:15 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:22:21,359] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '5f965dad-5d86-4c98-b6d1-6d98f5a8d6d4', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '5f965dad-5d86-4c98-b6d1-6d98f5a8d6d4', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:22:20 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:22:26,681] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '31460670-9824-497d-8483-00f9d2774d55', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '31460670-9824-497d-8483-00f9d2774d55', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:22:26 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:22:31,900] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '9b39367c-3267-4112-8e75-4375c22a9d4e', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '9b39367c-3267-4112-8e75-4375c22a9d4e', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:22:31 GMT'}, 'RetryAttempts': 0}}
|[2020-02-13 14:22:37,104] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'b499ab84-2e15-4836-8fc7-73431da25953', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'b499ab84-2e15-4836-8fc7-73431da25953', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:22:36 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:22:42,331] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '75eb1f89-00bf-4874-a665-7872f98ec480', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '75eb1f89-00bf-4874-a665-7872f98ec480', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:22:42 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:22:47,516] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '02e998e6-1c7e-4224-9616-09047ed43070', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '02e998e6-1c7e-4224-9616-09047ed43070', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:22:47 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:22:52,946] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '0c9038c6-8420-4d78-b139-71de41951d10', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '0c9038c6-8420-4d78-b139-71de41951d10', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:22:52 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:22:58,127] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '3876706b-3aca-476b-b5e0-cfc9effdf773', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '3876706b-3aca-476b-b5e0-cfc9effdf773', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:22:58 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:23:03,406] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '3d1739fc-d7f1-4e3e-af1f-13a07d580ab4', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '3d1739fc-d7f1-4e3e-af1f-13a07d580ab4', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:23:02 GMT'}, 'RetryAttempts': 0}}
|[2020-02-13 14:23:08,642] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '34b124f6-0967-4572-928b-dc1b3397064c', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '34b124f6-0967-4572-928b-dc1b3397064c', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:23:07 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:23:13,836] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'f365b5a5-ee92-4045-b297-3721379057d5', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'f365b5a5-ee92-4045-b297-3721379057d5', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:23:13 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:23:19,031] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '3c18acc3-256b-4447-831f-4e4d8fd2d1bf', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '3c18acc3-256b-4447-831f-4e4d8fd2d1bf', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:23:18 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:23:24,224] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '509a182e-e929-4c64-8648-a143d17ee038', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '509a182e-e929-4c64-8648-a143d17ee038', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:23:23 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:23:29,410] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'a003ceea-45bc-462d-aa1a-5b9bba1f0d50', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'a003ceea-45bc-462d-aa1a-5b9bba1f0d50', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:23:29 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:23:34,617] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '8c3e9d26-74d7-4520-9cd5-dc7efa58be90', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '8c3e9d26-74d7-4520-9cd5-dc7efa58be90', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:23:34 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:23:39,797] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'f4f0b913-f0fc-43fa-b6a2-dd6dffc81991', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'f4f0b913-f0fc-43fa-b6a2-dd6dffc81991', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:23:39 GMT'}, 'RetryAttempts': 0}}
|[2020-02-13 14:23:45,009] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '9784f6a4-2c92-483c-9b91-66e1916dde54', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '9784f6a4-2c92-483c-9b91-66e1916dde54', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:23:44 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:23:50,218] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '68616a40-0db8-4c6a-8f3c-5eb21e6ce6d7', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '68616a40-0db8-4c6a-8f3c-5eb21e6ce6d7', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:23:49 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:23:56,391] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '5076e03e-0a71-49f4-9545-8216d640a972', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '5076e03e-0a71-49f4-9545-8216d640a972', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:23:56 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:24:01,584] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '46188f6f-6b4c-4705-a400-ee6e36be88af', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '46188f6f-6b4c-4705-a400-ee6e36be88af', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:24:01 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:24:06,769] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'aa222871-33f7-4a16-b617-a0bc795ff936', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'aa222871-33f7-4a16-b617-a0bc795ff936', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:24:06 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:24:11,949] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '953c99af-5105-406b-965e-b190a4fdc40a', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '953c99af-5105-406b-965e-b190a4fdc40a', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:24:11 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:24:17,143] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '6d96055f-d67b-484a-83f2-e196778e6a73', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '6d96055f-d67b-484a-83f2-e196778e6a73', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:24:16 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:24:22,316] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '19c8ee58-ddac-42bd-9543-e235bcc505ce', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '19c8ee58-ddac-42bd-9543-e235bcc505ce', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:24:21 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:24:27,511] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '3e48b278-bfca-4532-a0ee-bed7ac3b6278', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '3e48b278-bfca-4532-a0ee-bed7ac3b6278', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:24:27 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:24:32,720] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '1dc72173-3a1c-4c42-907b-5d47c3684113', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '1dc72173-3a1c-4c42-907b-5d47c3684113', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:24:32 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:24:37,903] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '16750f66-f3e7-4eae-b680-1c218fb79668', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '16750f66-f3e7-4eae-b680-1c218fb79668', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:24:37 GMT'}, 'RetryAttempts': 0}}
|[2020-02-13 14:24:43,109] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'b6f318e5-942e-414a-85a9-59ce33932b37', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'b6f318e5-942e-414a-85a9-59ce33932b37', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:24:42 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:24:48,290] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '6e370366-1582-44c1-87da-5e5fa04aac96', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '6e370366-1582-44c1-87da-5e5fa04aac96', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:24:48 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:24:53,505] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '4c015c57-d57a-4f0e-b7fa-7ce8d957be66', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '4c015c57-d57a-4f0e-b7fa-7ce8d957be66', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:24:53 GMT'}, 'RetryAttempts': 0}}
|[2020-02-13 14:24:58,689] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'e1b0b934-54be-482b-86e8-f916b8ec7641', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'e1b0b934-54be-482b-86e8-f916b8ec7641', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:24:58 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:25:03,887] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '5940aaff-ef07-457e-aa12-0bfca6d9c360', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '5940aaff-ef07-457e-aa12-0bfca6d9c360', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:25:03 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:25:09,111] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '00872658-b214-427c-a15a-ad5d75a03569', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '00872658-b214-427c-a15a-ad5d75a03569', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:25:08 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:25:14,294] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '0cda17e0-9d27-4a42-8cbf-7bd16bac6f1a', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '0cda17e0-9d27-4a42-8cbf-7bd16bac6f1a', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:25:13 GMT'}, 'RetryAttempts': 0}}
|[2020-02-13 14:25:19,515] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'fa257468-a834-4287-b27e-ea0d63764617', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'fa257468-a834-4287-b27e-ea0d63764617', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:25:19 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:25:24,882] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '181879e3-9be2-49d9-ac3b-36be07b9743a', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '181879e3-9be2-49d9-ac3b-36be07b9743a', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:25:23 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:25:30,067] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '21c4f976-bb34-4b03-b806-0ac41ee6c64a', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '21c4f976-bb34-4b03-b806-0ac41ee6c64a', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:25:30 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:25:35,286] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '2ffa976c-f8ef-4b41-abf1-1097847e288a', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '2ffa976c-f8ef-4b41-abf1-1097847e288a', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:25:34 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:25:40,476] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'bdf74637-0603-4918-b2d3-9840af69b560', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'bdf74637-0603-4918-b2d3-9840af69b560', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:25:39 GMT'}, 'RetryAttempts': 0}}
|[2020-02-13 14:25:45,669] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '4456502f-bdb0-430f-ad11-17b81d4338a3', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '4456502f-bdb0-430f-ad11-17b81d4338a3', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:25:45 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:25:50,855] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '0495a2bf-a496-4f2e-9174-41b2d646a1f9', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '0495a2bf-a496-4f2e-9174-41b2d646a1f9', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:25:50 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:25:56,043] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '5e82b59b-59ca-4f15-b9bd-cb87447943c1', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '5e82b59b-59ca-4f15-b9bd-cb87447943c1', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:25:55 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:26:01,238] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '66506f82-c430-4571-be34-cfacf73ebb28', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '66506f82-c430-4571-be34-cfacf73ebb28', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:26:00 GMT'}, 'RetryAttempts': 0}}
|[2020-02-13 14:26:06,426] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '18dd299a-a49a-4fa2-b3cd-3300c270644d', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '18dd299a-a49a-4fa2-b3cd-3300c270644d', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:26:05 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:26:11,618] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '6bc62688-2c62-4772-95a9-e5845c5c6b67', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '6bc62688-2c62-4772-95a9-e5845c5c6b67', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:26:10 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:26:16,815] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'a86d0483-eee6-4c39-aaeb-f9d0bcd13176', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'a86d0483-eee6-4c39-aaeb-f9d0bcd13176', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:26:15 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:26:22,000] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '603753b0-dd9c-4bd1-8354-7fb0b3bcb0ca', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '603753b0-dd9c-4bd1-8354-7fb0b3bcb0ca', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:26:21 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:26:27,215] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'a700e95c-5d0a-4071-8f80-8b53763d9173', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'a700e95c-5d0a-4071-8f80-8b53763d9173', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:26:27 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:26:32,419] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'd2a222f0-5d4f-4b20-9d3f-1484dd58312a', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'd2a222f0-5d4f-4b20-9d3f-1484dd58312a', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:26:32 GMT'}, 'RetryAttempts': 0}}
|[2020-02-13 14:26:37,608] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '7a0d58ae-847b-4fa9-a054-0a6024a81b03', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '7a0d58ae-847b-4fa9-a054-0a6024a81b03', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:26:37 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:26:42,795] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'e7dbda46-ecb7-46c4-8766-9cafe125a917', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'e7dbda46-ecb7-46c4-8766-9cafe125a917', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:26:42 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:26:47,989] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '989fb689-7645-4fa9-b30c-b375780911d4', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '989fb689-7645-4fa9-b30c-b375780911d4', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:26:47 GMT'}, 'RetryAttempts': 0}}
|[2020-02-13 14:26:53,214] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '0fa8a29a-d226-4b32-8c2f-ce0926cca4fe', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '0fa8a29a-d226-4b32-8c2f-ce0926cca4fe', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:26:52 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:26:58,397] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '753aac20-a423-4a70-92c6-029512c64b5c', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '753aac20-a423-4a70-92c6-029512c64b5c', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:26:58 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:27:03,592] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '2fb5477a-2da7-4724-a13d-eecc85bdfad3', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '2fb5477a-2da7-4724-a13d-eecc85bdfad3', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:27:03 GMT'}, 'RetryAttempts': 0}}
|[2020-02-13 14:27:08,766] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'a00f2665-e579-4daa-a648-5813ce64c22e', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'a00f2665-e579-4daa-a648-5813ce64c22e', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:27:08 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:27:13,951] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '768f96c0-cfb4-45e4-8f35-9e787a025bb7', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '768f96c0-cfb4-45e4-8f35-9e787a025bb7', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:27:13 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:27:19,135] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'd8fc0004-9ca1-4e48-8ca7-61cde409d64a', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'd8fc0004-9ca1-4e48-8ca7-61cde409d64a', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:27:18 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:27:24,312] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'fe2279a1-37a0-49d2-9a4c-f9efbd1e9cb2', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'fe2279a1-37a0-49d2-9a4c-f9efbd1e9cb2', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:27:23 GMT'}, 'RetryAttempts': 0}}
|[2020-02-13 14:27:29,488] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'e14fcc28-ca42-4933-bb14-3ebed39c3782', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'e14fcc28-ca42-4933-bb14-3ebed39c3782', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:27:28 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:27:34,672] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'b75aa3be-022b-4005-bd99-9d1874c67544', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'b75aa3be-022b-4005-bd99-9d1874c67544', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:27:34 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:27:39,866] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '19227710-fcfa-4ed6-954c-8a871d195c97', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '19227710-fcfa-4ed6-954c-8a871d195c97', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:27:38 GMT'}, 'RetryAttempts': 0}}
|[2020-02-13 14:27:45,064] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'cf7dcc62-8e7b-4526-b525-7a05fa78de37', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'cf7dcc62-8e7b-4526-b525-7a05fa78de37', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:27:44 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:27:50,599] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'feecb227-87af-4695-9e24-15ae267c3769', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'feecb227-87af-4695-9e24-15ae267c3769', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:27:49 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:27:55,790] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'cad7d979-7be9-4a7f-9992-11f44e07c066', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'cad7d979-7be9-4a7f-9992-11f44e07c066', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:27:55 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:28:00,971] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '5f10b5fe-2fc6-456c-afba-c79fc892f4e9', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '5f10b5fe-2fc6-456c-afba-c79fc892f4e9', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:28:00 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:28:06,158] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'fd00492d-cddf-4052-8cbb-a0d2493df791', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'fd00492d-cddf-4052-8cbb-a0d2493df791', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:28:05 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:28:11,342] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'c737cc0a-9657-4fd9-8b2c-86edfda129cd', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'c737cc0a-9657-4fd9-8b2c-86edfda129cd', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:28:10 GMT'}, 'RetryAttempts': 0}}
|[2020-02-13 14:28:16,528] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '3851e8e2-3884-44da-90dc-0656431a6430', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '3851e8e2-3884-44da-90dc-0656431a6430', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:28:16 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:28:21,706] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '8619ff93-0331-4658-afb1-ca100a79aa99', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '8619ff93-0331-4658-afb1-ca100a79aa99', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:28:20 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:28:26,887] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'c4e657c2-ba79-461a-83e3-40a74640f718', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'c4e657c2-ba79-461a-83e3-40a74640f718', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:28:26 GMT'}, 'RetryAttempts': 0}}
|[2020-02-13 14:28:32,060] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '37684bf6-8fed-465f-9cdb-6d2e93a5cf91', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '37684bf6-8fed-465f-9cdb-6d2e93a5cf91', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:28:31 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:28:37,242] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'cc3e0e33-94f0-4514-b807-9ccf0d1f0d41', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'cc3e0e33-94f0-4514-b807-9ccf0d1f0d41', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:28:36 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:28:42,427] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'd37cf3bc-694b-48b3-953c-cc3e49525b33', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'd37cf3bc-694b-48b3-953c-cc3e49525b33', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:28:41 GMT'}, 'RetryAttempts': 0}}
|[2020-02-13 14:28:47,608] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'cddc65d5-2eff-4cdd-8b6d-b89cd1622f5d', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'cddc65d5-2eff-4cdd-8b6d-b89cd1622f5d', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:28:46 GMT'}, 'RetryAttempts': 0}}
|[2020-02-13 14:28:52,912] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'af0a8462-96f9-4dfa-8eec-2c405b8b5781', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'af0a8462-96f9-4dfa-8eec-2c405b8b5781', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:28:52 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:28:58,084] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '659182c5-bf0e-48b8-8e46-bff8b3825f4e', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '659182c5-bf0e-48b8-8e46-bff8b3825f4e', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:28:57 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:29:03,294] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '10fd8daa-fc9a-4808-a105-20848aac17ac', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '10fd8daa-fc9a-4808-a105-20848aac17ac', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:29:02 GMT'}, 'RetryAttempts': 0}}
|[2020-02-13 14:29:08,481] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '7e1e2c13-c08a-470b-b89c-f12f83abf9e6', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '7e1e2c13-c08a-470b-b89c-f12f83abf9e6', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:29:08 GMT'}, 'RetryAttempts': 0}}
/[2020-02-13 14:29:13,652] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'e0ce8967-74a6-4344-9714-84967667e81a', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'e0ce8967-74a6-4344-9714-84967667e81a', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:29:13 GMT'}, 'RetryAttempts': 0}}
-[2020-02-13 14:29:18,863] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'bd3aede9-40cc-4240-b310-04104b380b1a', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'bd3aede9-40cc-4240-b310-04104b380b1a', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:29:18 GMT'}, 'RetryAttempts': 0}}
\[2020-02-13 14:29:24,561] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'EndpointStatus': 'Creating', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': 'efe518ff-2e38-4b47-8315-8e9f4638e216', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'efe518ff-2e38-4b47-8315-8e9f4638e216', 'content-type': 'application/x-amz-json-1.1', 'content-length': '297', 'date': 'Thu, 13 Feb 2020 22:29:23 GMT'}, 'RetryAttempts': 0}}
|[2020-02-13 14:29:29,780] DEBUG - AWS describe endpoint response: {'EndpointName': 'bobo-keras-toxic', 'EndpointArn': 'arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic', 'EndpointConfigName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'ProductionVariants': [{'VariantName': 'bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94', 'DeployedImages': [{'SpecifiedImage': '192023623294.dkr.ecr.us-west-2.amazonaws.com/toxiccommentclassification-sagemaker:20200213141504_137C94', 'ResolvedImage': '192023623294.dkr.ecr.us-west-2.amazonaws.com/[email protected]:80a02ccbe0d18d8b70f1d12ce17ead749932c1b827e2eb1ccdada4e043cd4035', 'ResolutionTime': datetime.datetime(2020, 2, 13, 14, 21, 15, 101000, tzinfo=tzlocal())}], 'CurrentWeight': 1.0, 'DesiredWeight': 1.0, 'CurrentInstanceCount': 1, 'DesiredInstanceCount': 1}], 'EndpointStatus': 'InService', 'CreationTime': datetime.datetime(2020, 2, 13, 14, 21, 13, 367000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2020, 2, 13, 14, 29, 28, 820000, tzinfo=tzlocal()), 'ResponseMetadata': {'RequestId': '404dcf00-f9ad-4ac3-9b09-6aa616f8f76b', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '404dcf00-f9ad-4ac3-9b09-6aa616f8f76b', 'content-type': 'application/x-amz-json-1.1', 'content-length': '838', 'date': 'Thu, 13 Feb 2020 22:29:29 GMT'}, 'RetryAttempts': 0}}
Successfully created AWS Sagemaker deployment keras-toxic
{
  "namespace": "bobo",
  "name": "keras-toxic",
  "spec": {
    "bentoName": "ToxicCommentClassification",
    "bentoVersion": "20200213141504_137C94",
    "operator": "AWS_SAGEMAKER",
    "sagemakerOperatorConfig": {
      "region": "us-west-2",
      "instanceType": "ml.m4.xlarge",
      "instanceCount": 1,
      "apiName": "predict"
    }
  },
  "state": {
    "state": "RUNNING",
    "infoJson": {
      "EndpointName": "bobo-keras-toxic",
      "EndpointArn": "arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic",
      "EndpointConfigName": "bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94",
      "ProductionVariants": [
        {
          "VariantName": "bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94",
          "DeployedImages": [
            {
              "SpecifiedImage": "192023623294.dkr.ecr.us-west-2.amazonaws.com/toxiccommentclassification-sagemaker:20200213141504_137C94",
              "ResolvedImage": "192023623294.dkr.ecr.us-west-2.amazonaws.com/[email protected]:80a02ccbe0d18d8b70f1d12ce17ead749932c1b827e2eb1ccdada4e043cd4035",
              "ResolutionTime": "2020-02-13 14:21:15.101000-08:00"
            }
          ],
          "CurrentWeight": 1.0,
          "DesiredWeight": 1.0,
          "CurrentInstanceCount": 1,
          "DesiredInstanceCount": 1
        }
      ],
      "EndpointStatus": "InService",
      "CreationTime": "2020-02-13 14:21:13.367000-08:00",
      "LastModifiedTime": "2020-02-13 14:29:28.820000-08:00",
      "ResponseMetadata": {
        "RequestId": "404dcf00-f9ad-4ac3-9b09-6aa616f8f76b",
        "HTTPStatusCode": 200,
        "HTTPHeaders": {
          "x-amzn-requestid": "404dcf00-f9ad-4ac3-9b09-6aa616f8f76b",
          "content-type": "application/x-amz-json-1.1",
          "content-length": "838",
          "date": "Thu, 13 Feb 2020 22:29:29 GMT"
        },
        "RetryAttempts": 0
      }
    },
    "timestamp": "2020-02-13T22:29:29.781050Z"
  },
  "createdAt": "2020-02-13T22:17:11.078696Z",
  "lastUpdatedAt": "2020-02-13T22:17:11.078732Z"
}

bentoml sagemaker list displays all deployed Sagemaker deployments

In [29]:
!bentoml sagemaker list
NAME         NAMESPACE    PLATFORM       BENTO_SERVICE                                     STATUS    AGE
keras-toxic  bobo         aws-sagemaker  ToxicCommentClassification:20200213141504_137C94  running   12 minutes and 44.12 seconds

bentoml sagemaker get retrieve the latest status of Sagemaker deployment

In [30]:
!bentoml sagemaker get keras-toxic
{
  "namespace": "bobo",
  "name": "keras-toxic",
  "spec": {
    "bentoName": "ToxicCommentClassification",
    "bentoVersion": "20200213141504_137C94",
    "operator": "AWS_SAGEMAKER",
    "sagemakerOperatorConfig": {
      "region": "us-west-2",
      "instanceType": "ml.m4.xlarge",
      "instanceCount": 1,
      "apiName": "predict"
    }
  },
  "state": {
    "state": "RUNNING",
    "infoJson": {
      "EndpointName": "bobo-keras-toxic",
      "EndpointArn": "arn:aws:sagemaker:us-west-2:192023623294:endpoint/bobo-keras-toxic",
      "EndpointConfigName": "bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94",
      "ProductionVariants": [
        {
          "VariantName": "bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94",
          "DeployedImages": [
            {
              "SpecifiedImage": "192023623294.dkr.ecr.us-west-2.amazonaws.com/toxiccommentclassification-sagemaker:20200213141504_137C94",
              "ResolvedImage": "192023623294.dkr.ecr.us-west-2.amazonaws.com/[email protected]:80a02ccbe0d18d8b70f1d12ce17ead749932c1b827e2eb1ccdada4e043cd4035",
              "ResolutionTime": "2020-02-13 14:21:15.101000-08:00"
            }
          ],
          "CurrentWeight": 1.0,
          "DesiredWeight": 1.0,
          "CurrentInstanceCount": 1,
          "DesiredInstanceCount": 1
        }
      ],
      "EndpointStatus": "InService",
      "CreationTime": "2020-02-13 14:21:13.367000-08:00",
      "LastModifiedTime": "2020-02-13 14:29:28.820000-08:00",
      "ResponseMetadata": {
        "RequestId": "1d97799c-7cf1-4dfb-94ef-bf337575d48c",
        "HTTPStatusCode": 200,
        "HTTPHeaders": {
          "x-amzn-requestid": "1d97799c-7cf1-4dfb-94ef-bf337575d48c",
          "content-type": "application/x-amz-json-1.1",
          "content-length": "838",
          "date": "Thu, 13 Feb 2020 22:29:56 GMT"
        },
        "RetryAttempts": 0
      }
    },

  },
  "createdAt": "2020-02-13T22:17:11.078696Z",
  "lastUpdatedAt": "2020-02-13T22:17:11.078732Z"
}

Validate and test Sagemaker deployment with sample data

In [31]:
!aws sagemaker-runtime invoke-endpoint --endpoint-name bobo-keras-toxic \
--body '[{"comment_text": "bad terrible"}]' --content-type application/json output.json && cat output.json
{
    "ContentType": "application/json",
    "InvokedProductionVariant": "bobo-keras-toxic-ToxicCommentClassifi-20200213141504-137C94"
}
["toxic"]

bentoml sagemaker delete will remove Sagmaker deployment and related resources

In [32]:
!bentoml sagemaker delete keras-toxic
Successfully deleted AWS Sagemaker deployment "keras-toxic"