Titanic Survival Prediction with LightBGM

BentoML makes moving trained ML models to production easy:

  • Package models trained with any ML framework and reproduce them for model serving in production
  • Deploy anywhere for online API serving or offline batch serving
  • High-Performance API model server with adaptive micro-batching support
  • Central hub for managing models and deployment process via Web UI and APIs
  • Modular and flexible design making it adaptable to your infrastrcuture

BentoML is a framework for serving, managing, and deploying machine learning models. It is aiming to bridge the gap between Data Science and DevOps, and enable teams to deliver prediction services in a fast, repeatable, and scalable way.

Before reading this example project, be sure to check out the Getting started guide to learn about the basic concepts in BentoML.

This notebook is demonstrating how to package and serve LightBGM model for production using BentoML.

Impression

In [1]:
%reload_ext autoreload
%autoreload 2
%matplotlib inline

import warnings
warnings.filterwarnings("ignore")
In [3]:
!pip install -q bentoml "lightgbm==2.3.1" numpy pandas
Installing collected packages: lightgbm
Successfully installed lightgbm-2.3.1
In [2]:
import pandas as pd
import numpy as np
import lightgbm as lgb
from sklearn.model_selection import train_test_split
import bentoml

Prepare Dataset

download dataset from https://www.kaggle.com/c/titanic/data

In [3]:
!mkdir data
!curl https://raw.githubusercontent.com/agconti/kaggle-titanic/master/data/train.csv -o ./data/train.csv
!curl https://raw.githubusercontent.com/agconti/kaggle-titanic/master/data/test.csv -o ./data/test.csv
mkdir: data: File exists
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 60302  100 60302    0     0   148k      0 --:--:-- --:--:-- --:--:--  147k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 28210  100 28210    0     0  94347      0 --:--:-- --:--:-- --:--:-- 94347
In [4]:
train_df = pd.read_csv('./data/train.csv')
test_df = pd.read_csv('./data/test.csv')
train_df.head()
Out[4]:
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked
0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.2500 NaN S
1 2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Th... female 38.0 1 0 PC 17599 71.2833 C85 C
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 STON/O2. 3101282 7.9250 NaN S
3 4 1 1 Futrelle, Mrs. Jacques Heath (Lily May Peel) female 35.0 1 0 113803 53.1000 C123 S
4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.0500 NaN S
In [5]:
y = train_df.pop('Survived')
cols = ['Pclass', 'Age', 'Fare', 'SibSp', 'Parch']
X_train, X_test, y_train, y_test = train_test_split(train_df[cols], 
                                                    y, 
                                                    test_size=0.2, 
                                                    random_state=42)
In [6]:
# Create an LGBM dataset for training
train_data = lgb.Dataset(data=X_train[cols],
                        label=y_train)

# Create an LGBM dataset from the test
test_data = lgb.Dataset(data=X_test[cols],
                        label=y_test)

Model Training

In [7]:
lgb_params = {
    'boosting': 'dart',          # dart (drop out trees) often performs better
    'application': 'binary',     # Binary classification
    'learning_rate': 0.05,       # Learning rate, controls size of a gradient descent step
    'min_data_in_leaf': 20,      # Data set is quite small so reduce this a bit
    'feature_fraction': 0.7,     # Proportion of features in each boost, controls overfitting
    'num_leaves': 41,            # Controls size of tree since LGBM uses leaf wise splits
    'metric': 'binary_logloss',  # Area under ROC curve as the evaulation metric
    'drop_rate': 0.15
              }

evaluation_results = {}
model = lgb.train(train_set=train_data,
                params=lgb_params,
                valid_sets=[train_data, test_data], 
                valid_names=['Train', 'Test'],
                evals_result=evaluation_results,
                num_boost_round=500,
                early_stopping_rounds=100,
                verbose_eval=20
                )
[20]	Train's binary_logloss: 0.55215	Test's binary_logloss: 0.587358
[40]	Train's binary_logloss: 0.510164	Test's binary_logloss: 0.559348
[60]	Train's binary_logloss: 0.500602	Test's binary_logloss: 0.551635
[80]	Train's binary_logloss: 0.490215	Test's binary_logloss: 0.547154
[100]	Train's binary_logloss: 0.486812	Test's binary_logloss: 0.547076
[120]	Train's binary_logloss: 0.479242	Test's binary_logloss: 0.542552
[140]	Train's binary_logloss: 0.469847	Test's binary_logloss: 0.539319
[160]	Train's binary_logloss: 0.471384	Test's binary_logloss: 0.542278
[180]	Train's binary_logloss: 0.453052	Test's binary_logloss: 0.535512
[200]	Train's binary_logloss: 0.442048	Test's binary_logloss: 0.533921
[220]	Train's binary_logloss: 0.436788	Test's binary_logloss: 0.534261
[240]	Train's binary_logloss: 0.427196	Test's binary_logloss: 0.532026
[260]	Train's binary_logloss: 0.420145	Test's binary_logloss: 0.531791
[280]	Train's binary_logloss: 0.413336	Test's binary_logloss: 0.527412
[300]	Train's binary_logloss: 0.406546	Test's binary_logloss: 0.529314
[320]	Train's binary_logloss: 0.402753	Test's binary_logloss: 0.525075
[340]	Train's binary_logloss: 0.39979	Test's binary_logloss: 0.523438
[360]	Train's binary_logloss: 0.403024	Test's binary_logloss: 0.525361
[380]	Train's binary_logloss: 0.398387	Test's binary_logloss: 0.528122
[400]	Train's binary_logloss: 0.394841	Test's binary_logloss: 0.529159
[420]	Train's binary_logloss: 0.390478	Test's binary_logloss: 0.528173
[440]	Train's binary_logloss: 0.384254	Test's binary_logloss: 0.526504
[460]	Train's binary_logloss: 0.381594	Test's binary_logloss: 0.525863
[480]	Train's binary_logloss: 0.371362	Test's binary_logloss: 0.527399
[500]	Train's binary_logloss: 0.369978	Test's binary_logloss: 0.525418
In [8]:
test_df['pred'] = model.predict(test_df[cols])
test_df[['Pclass', 'Age', 'Fare', 'SibSp', 'Parch','pred']].iloc[10:].head(2)
Out[8]:
Pclass Age Fare SibSp Parch pred
10 3 NaN 7.8958 0 0 0.052353
11 1 46.0 26.0000 0 0 0.308877

Create BentoService for model serving

In [9]:
%%writefile lightbgm_titanic_bento_service.py

import lightgbm as lgb

import bentoml
from bentoml.frameworks.lightgbm import LightGBMModelArtifact
from bentoml.adapters import DataframeInput

@bentoml.artifacts([LightGBMModelArtifact('model')])
@bentoml.env(pip_packages=['lightgbm'])
class TitanicSurvivalPredictionService(bentoml.BentoService):
    
    @bentoml.api(input=DataframeInput(), batch=True)
    def predict(self, df):
        data = df[['Pclass', 'Age', 'Fare', 'SibSp', 'Parch']]
        return self.artifacts.model.predict(data)
Overwriting lightbgm_titanic_bento_service.py

Save BentoML service archive

In [10]:
# 1) import the custom BentoService defined above
from lightbgm_titanic_bento_service import TitanicSurvivalPredictionService

# 2) `pack` it with required artifacts
bento_service = TitanicSurvivalPredictionService()
bento_service.pack('model', model)

# 3) save your BentoSerivce
saved_path = bento_service.save()
[2020-09-22 12:33:34,555] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.
[2020-09-22 12:33:35,176] INFO - Using default docker base image: `None` specified inBentoML config file or env var. User must make sure that the docker base image either has Python 3.7 or conda installed.
[2020-09-22 12:33:35,181] WARNING - pip package requirement lightgbm already exist
[2020-09-22 12:33:36,636] INFO - Detected non-PyPI-released BentoML installed, copying local BentoML modulefiles to target saved bundle path..
warning: no previously-included files matching '*~' found anywhere in distribution
warning: no previously-included files matching '*.pyo' found anywhere in distribution
warning: no previously-included files matching '.git' found anywhere in distribution
warning: no previously-included files matching '.ipynb_checkpoints' found anywhere in distribution
warning: no previously-included files matching '__pycache__' found anywhere in distribution
no previously-included directories found matching 'e2e_tests'
no previously-included directories found matching 'tests'
no previously-included directories found matching 'benchmark'
UPDATING BentoML-0.9.0rc0+3.gcebf2015/bentoml/_version.py
set BentoML-0.9.0rc0+3.gcebf2015/bentoml/_version.py to '0.9.0.pre+3.gcebf2015'
[2020-09-22 12:33:40,777] INFO - BentoService bundle 'TitanicSurvivalPredictionService:20200922123336_3D25BB' saved to: /Users/bozhaoyu/bentoml/repository/TitanicSurvivalPredictionService/20200922123336_3D25BB

REST API Model Serving

To start a REST API model server with the BentoService saved above, use the bentoml serve command:

In [11]:
!bentoml serve TitanicSurvivalPredictionService:latest
[2020-09-22 12:34:00,840] INFO - Getting latest version TitanicSurvivalPredictionService:20200922123336_3D25BB
[2020-09-22 12:34:00,842] INFO - Starting BentoML API server in development mode..
[2020-09-22 12:34:01,398] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.
[2020-09-22 12:34:01,440] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015
[2020-09-22 12:34:03,740] INFO - Using default docker base image: `None` specified inBentoML config file or env var. User must make sure that the docker base image either has Python 3.7 or conda installed.
[2020-09-22 12:34:03,780] WARNING - pip package requirement lightgbm already exist
 * Serving Flask app "TitanicSurvivalPredictionService" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
[2020-09-22 12:34:13,719] INFO - {'service_name': 'TitanicSurvivalPredictionService', 'service_version': '20200922123336_3D25BB', 'api': 'predict', 'task': {'data': {}, 'task_id': '35f9e40e-d162-4994-a8e2-6eddaa947572', 'batch': 1, 'http_headers': (('Host', 'localhost:5000'), ('User-Agent', 'curl/7.65.3'), ('Accept', '*/*'), ('Content-Type', 'application/json'), ('Content-Length', '63'))}, 'result': {'data': '[0.8073128889994113]', 'http_status': 200, 'http_headers': (('Content-Type', 'application/json'),)}, 'request_id': '35f9e40e-d162-4994-a8e2-6eddaa947572'}
127.0.0.1 - - [22/Sep/2020 12:34:13] "POST /predict HTTP/1.1" 200 -
WARNING: Logging before flag parsing goes to stderr.
I0922 12:34:13.721330 4449889728 _internal.py:122] 127.0.0.1 - - [22/Sep/2020 12:34:13] "POST /predict HTTP/1.1" 200 -
^C

If you are running this notebook from Google Colab, you can start the dev server with --run-with-ngrok option, to gain acccess to the API endpoint via a public endpoint managed by ngrok:

In [ ]:
!bentoml serve TitanicSurvivalPredictionService:latest --run-w8th-ngrok

Open http://127.0.0.1:5000 to see more information about the REST APIs server in your browser.

Send prediction requeset to the REST API server

Navigate to parent directory of the notebook(so you have reference to the test.jpg image), and run the following curl command to send the image to REST API server and get a prediction result:

curl -i \
--header "Content-Type: application/json" \
--request POST \
--data '[{"Pclass": 1, "Age": 30, "Fare": 200, "SibSp": 1, "Parch": 0}]' \
localhost:5000/predict

Containerize model server with Docker

One common way of distributing this model API server for production deployment, is via Docker containers. And BentoML provides a convenient way to do that.

Note that docker is not available in Google Colab. You will need to download and run this notebook locally to try out this containerization with docker feature.

If you already have docker configured, simply run the follow command to product a docker container serving the IrisClassifier prediction service created above:

In [12]:
!bentoml containerize TitanicSurvivalPredictionService:latest
[2020-09-22 12:34:57,017] INFO - Getting latest version TitanicSurvivalPredictionService:20200922123336_3D25BB
Found Bento: /Users/bozhaoyu/bentoml/repository/TitanicSurvivalPredictionService/20200922123336_3D25BB
[2020-09-22 12:34:57,057] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.
[2020-09-22 12:34:57,077] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015
Tag not specified, using tag parsed from BentoService: 'titanicsurvivalpredictionservice:20200922123336_3D25BB'
Building Docker image titanicsurvivalpredictionservice:20200922123336_3D25BB from TitanicSurvivalPredictionService:latest 
-we in here
processed docker file
(None, None)
root in create archive /Users/bozhaoyu/bentoml/repository/TitanicSurvivalPredictionService/20200922123336_3D25BB ['Dockerfile', 'MANIFEST.in', 'README.md', 'TitanicSurvivalPredictionService', 'TitanicSurvivalPredictionService/__init__.py', 'TitanicSurvivalPredictionService/__pycache__', 'TitanicSurvivalPredictionService/__pycache__/lightbgm_titanic_bento_service.cpython-37.pyc', 'TitanicSurvivalPredictionService/artifacts', 'TitanicSurvivalPredictionService/artifacts/__init__.py', 'TitanicSurvivalPredictionService/artifacts/model.txt', 'TitanicSurvivalPredictionService/bentoml.yml', 'TitanicSurvivalPredictionService/lightbgm_titanic_bento_service.py', 'bentoml-init.sh', 'bentoml.yml', 'bundled_pip_dependencies', 'bundled_pip_dependencies/BentoML-0.9.0rc0+3.gcebf2015.tar.gz', 'docker-entrypoint.sh', 'environment.yml', 'python_version', 'requirements.txt', 'setup.py']
about to build
about to upgrade params
check each param and update
if use config proxy
if buildargs
if shmsize
if labels
if cache from
if target
if network_mode
if squash
if extra hosts is not None
if platform is not None
if isolcation is not None
if context is not None
setting auth {'Content-Type': 'application/tar'}
\docker build <tempfile._TemporaryFileWrapper object at 0x7fa9a1db3dd8> {'t': 'titanicsurvivalpredictionservice:20200922123336_3D25BB', 'remote': None, 'q': False, 'nocache': False, 'rm': False, 'forcerm': False, 'pull': False, 'dockerfile': (None, None)}
/docker response <Response [200]>
context closes
print responses
Step 1/15 : FROM bentoml/model-server:0.9.0.pre
 ---> a25066aa8b0e
Step 2/15 : ARG EXTRA_PIP_INSTALL_ARGS=
 ---> Using cache
 ---> fc6e47d06522
Step 3/15 : ENV EXTRA_PIP_INSTALL_ARGS $EXTRA_PIP_INSTALL_ARGS
 ---> Using cache
 ---> db8172e98571
Step 4/15 : COPY environment.yml requirements.txt setup.sh* bentoml-init.sh python_version* /bento/
/ ---> 50650c4e6206
Step 5/15 : WORKDIR /bento
| ---> Running in 642c7094626b
\ ---> 0364c28f2989
Step 6/15 : RUN chmod +x /bento/bentoml-init.sh
 ---> Running in d35196a8530b
/ ---> 0487bc2b9363
Step 7/15 : RUN if [ -f /bento/bentoml-init.sh ]; then bash -c /bento/bentoml-init.sh; fi
| ---> Running in 3f73a39c8f97
/+++ dirname /bento/bentoml-init.sh

++ cd /bento
++ pwd -P

+ SAVED_BUNDLE_PATH=/bento

+ cd /bento

+ '[' -f ./setup.sh ']'

+ '[' -f ./python_version ']'

++ cat ./python_version

+ PY_VERSION_SAVED=3.7.3
+ DESIRED_PY_VERSION=3.7

++ python -c 'import sys; print(f"{sys.version_info.major}.{sys.version_info.minor}")'

+ CURRENT_PY_VERSION=3.7

+ [[ 3.7 == \3\.\7 ]]
+ echo 'Python Version in docker base image 3.7 matches requirement python=3.7. Skipping.'
+ command -v conda
+ echo 'Updating conda base environment with environment.yml'
+ conda env update -n base -f ./environment.yml

Python Version in docker base image 3.7 matches requirement python=3.7. Skipping.
Updating conda base environment with environment.yml
|Collecting package metadata (repodata.json): ...working... 
/done
Solving environment: ...working... 
/done
|
Downloading and Extracting Packages
libffi-3.2.1         | 47 KB     |            |   0% 
libffi-3.2.1         | 47 KB     | ###4       |  34% 
libffi-3.2.1         | 47 KB     | ########## | 100% 

certifi-2020.6.20    | 151 KB    |            |   0% 
certifi-2020.6.20    | 151 KB    | ########## | 100% 

ca-certificates-2020 | 145 KB    |            |   0% 
ca-certificates-2020 | 145 KB    | ########## | 100% 

cffi-1.14.3          | 223 KB    |            |   0% 
cffi-1.14.3          | 223 KB    | ########## | 100% 
cffi-1.14.3          | 223 KB    | ########## | 100% 

pip-20.2.3           | 1.1 MB    |            |   0% 
pip-20.2.3           | 1.1 MB    | ####2      |  42% 
pip-20.2.3           | 1.1 MB    | ########## | 100% 
pip-20.2.3           | 1.1 MB    | ########## | 100% 

python_abi-3.7       | 4 KB      |            |   0% 
python_abi-3.7       | 4 KB      | ########## | 100% 

openssl-1.1.1h       | 2.1 MB    |            |   0% 
openssl-1.1.1h       | 2.1 MB    | ###3       |  33% 
openssl-1.1.1h       | 2.1 MB    | #########4 |  94% 
openssl-1.1.1h       | 2.1 MB    | ########## | 100% 
Preparing transaction: ...working... 
\done
Verifying transaction: ...working... 
-done
Executing transaction: ...working... 
-done
/#
# To activate this environment, use
#
#     $ conda activate base
#
# To deactivate an active environment, use
#
#     $ conda deactivate
/+ pip install -r ./requirements.txt --no-cache-dir

|Requirement already satisfied: bentoml==0.9.0.pre in /opt/conda/lib/python3.7/site-packages (from -r ./requirements.txt (line 1)) (0.9.0rc0)
|Collecting lightgbm==2.3.1
-  Downloading lightgbm-2.3.1-py2.py3-none-manylinux1_x86_64.whl (1.2 MB)
|Collecting pandas==0.24.2
  Downloading pandas-0.24.2-cp37-cp37m-manylinux1_x86_64.whl (10.1 MB)
/Requirement already satisfied: prometheus-client in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.8.0)
Requirement already satisfied: numpy in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.19.2)
Requirement already satisfied: protobuf>=3.6.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (3.13.0)
Requirement already satisfied: requests in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (2.24.0)
|Requirement already satisfied: boto3 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.15.2)
Requirement already satisfied: aiohttp in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (3.6.2)
Requirement already satisfied: docker in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (4.3.1)
Requirement already satisfied: multidict in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (4.7.6)
Requirement already satisfied: tabulate in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.8.7)
Requirement already satisfied: grpcio<=1.27.2 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.27.2)
Requirement already satisfied: cerberus in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.3.2)
Requirement already satisfied: alembic in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.4.3)
\Requirement already satisfied: sqlalchemy>=1.3.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.3.19)
Requirement already satisfied: python-json-logger in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.1.11)
Requirement already satisfied: certifi in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (2020.6.20)
Requirement already satisfied: flask in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.1.2)
Requirement already satisfied: py-zipkin in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.20.0)
Requirement already satisfied: packaging in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (20.4)
Requirement already satisfied: click>=7.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (7.1.2)
Requirement already satisfied: humanfriendly in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (8.2)
Requirement already satisfied: ruamel.yaml>=0.15.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.15.87)
Requirement already satisfied: sqlalchemy-utils<0.36.8 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.36.7)
-Requirement already satisfied: configparser in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (5.0.0)
Requirement already satisfied: psutil in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (5.7.2)
/Requirement already satisfied: gunicorn in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (20.0.4)
Requirement already satisfied: python-dateutil<3.0.0,>=2.7.3 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (2.8.1)
\Collecting scikit-learn
-  Downloading scikit_learn-0.23.2-cp37-cp37m-manylinux1_x86_64.whl (6.8 MB)
\Collecting scipy
  Downloading scipy-1.5.2-cp37-cp37m-manylinux1_x86_64.whl (25.9 MB)
|Collecting pytz>=2011k
  Downloading pytz-2020.1-py2.py3-none-any.whl (510 kB)
Requirement already satisfied: setuptools in /opt/conda/lib/python3.7/site-packages (from protobuf>=3.6.0->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (49.6.0.post20200814)
\Requirement already satisfied: six>=1.9 in /opt/conda/lib/python3.7/site-packages (from protobuf>=3.6.0->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.15.0)
Requirement already satisfied: idna<3,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (2.10)
Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.25.10)
Requirement already satisfied: botocore<1.19.0,>=1.18.2 in /opt/conda/lib/python3.7/site-packages (from boto3->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.18.2)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /opt/conda/lib/python3.7/site-packages (from boto3->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.10.0)
Requirement already satisfied: s3transfer<0.4.0,>=0.3.0 in /opt/conda/lib/python3.7/site-packages (from boto3->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.3.3)
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.5.1)
Requirement already satisfied: attrs>=17.3.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (20.2.0)
Requirement already satisfied: async-timeout<4.0,>=3.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (3.0.1)
Requirement already satisfied: websocket-client>=0.32.0 in /opt/conda/lib/python3.7/site-packages (from docker->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.57.0)
Requirement already satisfied: Mako in /opt/conda/lib/python3.7/site-packages (from alembic->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.1.3)
Requirement already satisfied: python-editor>=0.3 in /opt/conda/lib/python3.7/site-packages (from alembic->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.0.4)
-Requirement already satisfied: Werkzeug>=0.15 in /opt/conda/lib/python3.7/site-packages (from flask->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.0.1)
Requirement already satisfied: itsdangerous>=0.24 in /opt/conda/lib/python3.7/site-packages (from flask->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.1.0)
Requirement already satisfied: Jinja2>=2.10.1 in /opt/conda/lib/python3.7/site-packages (from flask->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (2.11.2)
Requirement already satisfied: thriftpy2>=0.4.0 in /opt/conda/lib/python3.7/site-packages (from py-zipkin->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.4.11)
Requirement already satisfied: pyparsing>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from packaging->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (2.4.7)
Collecting threadpoolctl>=2.0.0
/  Downloading threadpoolctl-2.1.0-py3-none-any.whl (12 kB)
|Collecting joblib>=0.11
  Downloading joblib-0.16.0-py3-none-any.whl (300 kB)
\Requirement already satisfied: typing-extensions>=3.7.4; python_version < "3.8" in /opt/conda/lib/python3.7/site-packages (from yarl<2.0,>=1.0->aiohttp->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (3.7.4.3)
Requirement already satisfied: MarkupSafe>=0.9.2 in /opt/conda/lib/python3.7/site-packages (from Mako->alembic->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.1.1)
Requirement already satisfied: ply<4.0,>=3.4 in /opt/conda/lib/python3.7/site-packages (from thriftpy2>=0.4.0->py-zipkin->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (3.11)
/Installing collected packages: scipy, threadpoolctl, joblib, scikit-learn, lightgbm, pytz, pandas
-Successfully installed joblib-0.16.0 lightgbm-2.3.1 pandas-0.24.2 pytz-2020.1 scikit-learn-0.23.2 scipy-1.5.2 threadpoolctl-2.1.0
\ ---> 6754314ac928
Step 8/15 : COPY . /bento
| ---> 003a2c5d3996
Step 9/15 : RUN if [ -d /bento/bundled_pip_dependencies ]; then pip install -U bundled_pip_dependencies/* ;fi
 ---> Running in 6ca4acc846c6
/Processing ./bundled_pip_dependencies/BentoML-0.9.0rc0+3.gcebf2015.tar.gz
/  Installing build dependencies: started
-  Installing build dependencies: finished with status 'done'
/  Getting requirements to build wheel: started
|  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
\    Preparing wheel metadata: finished with status 'done'
|Requirement already satisfied, skipping upgrade: boto3 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (1.15.2)
Requirement already satisfied, skipping upgrade: requests in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (2.24.0)
Requirement already satisfied, skipping upgrade: certifi in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (2020.6.20)
Requirement already satisfied, skipping upgrade: prometheus-client in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (0.8.0)
Requirement already satisfied, skipping upgrade: psutil in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (5.7.2)
Requirement already satisfied, skipping upgrade: sqlalchemy>=1.3.0 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (1.3.19)
Requirement already satisfied, skipping upgrade: py-zipkin in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (0.20.0)
Requirement already satisfied, skipping upgrade: protobuf>=3.6.0 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (3.13.0)
Requirement already satisfied, skipping upgrade: python-dateutil<3.0.0,>=2.7.3 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (2.8.1)
Requirement already satisfied, skipping upgrade: python-json-logger in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (0.1.11)
Requirement already satisfied, skipping upgrade: multidict in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (4.7.6)
Requirement already satisfied, skipping upgrade: aiohttp in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (3.6.2)
\Requirement already satisfied, skipping upgrade: docker in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (4.3.1)
Requirement already satisfied, skipping upgrade: alembic in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (1.4.3)
Requirement already satisfied, skipping upgrade: configparser in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (5.0.0)
Requirement already satisfied, skipping upgrade: humanfriendly in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (8.2)
Requirement already satisfied, skipping upgrade: cerberus in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (1.3.2)
Requirement already satisfied, skipping upgrade: packaging in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (20.4)
Requirement already satisfied, skipping upgrade: sqlalchemy-utils<0.36.8 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (0.36.7)
-Requirement already satisfied, skipping upgrade: numpy in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (1.19.2)
Requirement already satisfied, skipping upgrade: click>=7.0 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (7.1.2)
Requirement already satisfied, skipping upgrade: tabulate in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (0.8.7)
Requirement already satisfied, skipping upgrade: gunicorn in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (20.0.4)
Requirement already satisfied, skipping upgrade: ruamel.yaml>=0.15.0 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (0.15.87)
Requirement already satisfied, skipping upgrade: flask in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (1.1.2)
Requirement already satisfied, skipping upgrade: grpcio<=1.27.2 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (1.27.2)
Requirement already satisfied, skipping upgrade: jmespath<1.0.0,>=0.7.1 in /opt/conda/lib/python3.7/site-packages (from boto3->BentoML==0.9.0rc0+3.gcebf2015) (0.10.0)
Requirement already satisfied, skipping upgrade: s3transfer<0.4.0,>=0.3.0 in /opt/conda/lib/python3.7/site-packages (from boto3->BentoML==0.9.0rc0+3.gcebf2015) (0.3.3)
Requirement already satisfied, skipping upgrade: botocore<1.19.0,>=1.18.2 in /opt/conda/lib/python3.7/site-packages (from boto3->BentoML==0.9.0rc0+3.gcebf2015) (1.18.2)
Requirement already satisfied, skipping upgrade: chardet<4,>=3.0.2 in /opt/conda/lib/python3.7/site-packages (from requests->BentoML==0.9.0rc0+3.gcebf2015) (3.0.4)
Requirement already satisfied, skipping upgrade: idna<3,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests->BentoML==0.9.0rc0+3.gcebf2015) (2.10)
Requirement already satisfied, skipping upgrade: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests->BentoML==0.9.0rc0+3.gcebf2015) (1.25.10)
/Requirement already satisfied, skipping upgrade: thriftpy2>=0.4.0 in /opt/conda/lib/python3.7/site-packages (from py-zipkin->BentoML==0.9.0rc0+3.gcebf2015) (0.4.11)
Requirement already satisfied, skipping upgrade: six in /opt/conda/lib/python3.7/site-packages (from py-zipkin->BentoML==0.9.0rc0+3.gcebf2015) (1.15.0)
Requirement already satisfied, skipping upgrade: setuptools in /opt/conda/lib/python3.7/site-packages (from protobuf>=3.6.0->BentoML==0.9.0rc0+3.gcebf2015) (49.6.0.post20200814)
Requirement already satisfied, skipping upgrade: yarl<2.0,>=1.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->BentoML==0.9.0rc0+3.gcebf2015) (1.5.1)
Requirement already satisfied, skipping upgrade: attrs>=17.3.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->BentoML==0.9.0rc0+3.gcebf2015) (20.2.0)
Requirement already satisfied, skipping upgrade: async-timeout<4.0,>=3.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->BentoML==0.9.0rc0+3.gcebf2015) (3.0.1)
Requirement already satisfied, skipping upgrade: websocket-client>=0.32.0 in /opt/conda/lib/python3.7/site-packages (from docker->BentoML==0.9.0rc0+3.gcebf2015) (0.57.0)
Requirement already satisfied, skipping upgrade: python-editor>=0.3 in /opt/conda/lib/python3.7/site-packages (from alembic->BentoML==0.9.0rc0+3.gcebf2015) (1.0.4)
Requirement already satisfied, skipping upgrade: Mako in /opt/conda/lib/python3.7/site-packages (from alembic->BentoML==0.9.0rc0+3.gcebf2015) (1.1.3)
|Requirement already satisfied, skipping upgrade: pyparsing>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from packaging->BentoML==0.9.0rc0+3.gcebf2015) (2.4.7)
Requirement already satisfied, skipping upgrade: Jinja2>=2.10.1 in /opt/conda/lib/python3.7/site-packages (from flask->BentoML==0.9.0rc0+3.gcebf2015) (2.11.2)
Requirement already satisfied, skipping upgrade: Werkzeug>=0.15 in /opt/conda/lib/python3.7/site-packages (from flask->BentoML==0.9.0rc0+3.gcebf2015) (1.0.1)
Requirement already satisfied, skipping upgrade: itsdangerous>=0.24 in /opt/conda/lib/python3.7/site-packages (from flask->BentoML==0.9.0rc0+3.gcebf2015) (1.1.0)
Requirement already satisfied, skipping upgrade: ply<4.0,>=3.4 in /opt/conda/lib/python3.7/site-packages (from thriftpy2>=0.4.0->py-zipkin->BentoML==0.9.0rc0+3.gcebf2015) (3.11)
Requirement already satisfied, skipping upgrade: typing-extensions>=3.7.4; python_version < "3.8" in /opt/conda/lib/python3.7/site-packages (from yarl<2.0,>=1.0->aiohttp->BentoML==0.9.0rc0+3.gcebf2015) (3.7.4.3)
Requirement already satisfied, skipping upgrade: MarkupSafe>=0.9.2 in /opt/conda/lib/python3.7/site-packages (from Mako->alembic->BentoML==0.9.0rc0+3.gcebf2015) (1.1.1)
Building wheels for collected packages: BentoML
  Building wheel for BentoML (PEP 517): started
\  Building wheel for BentoML (PEP 517): finished with status 'done'
  Created wheel for BentoML: filename=BentoML-0.9.0rc0+3.gcebf2015-py3-none-any.whl size=3064091 sha256=78a6db2b339ef74013a744e6bed1e5e3603f3df140eb51c6140fc00bfc8586e3
  Stored in directory: /root/.cache/pip/wheels/a0/45/41/62152db705af4ff47e7a3d6abf6247986eef4aa1b94a58d3b9
Successfully built BentoML
\Installing collected packages: BentoML
  Attempting uninstall: BentoML
    Found existing installation: BentoML 0.9.0rc0
/    Uninstalling BentoML-0.9.0rc0:
\      Successfully uninstalled BentoML-0.9.0rc0
/Successfully installed BentoML-0.9.0rc0+3.gcebf2015
\ ---> 44b969a3203e
Step 10/15 : ENV PORT 5000
 ---> Running in e88926beae46
- ---> 9902f11d310e
Step 11/15 : EXPOSE $PORT
/ ---> Running in 9e9ed00cd096
| ---> 2f799ca8672c
Step 12/15 : COPY docker-entrypoint.sh /usr/local/bin/
- ---> 9ef1c0c8a751
Step 13/15 : RUN chmod +x /usr/local/bin/docker-entrypoint.sh
 ---> Running in 1f4e2c667944
| ---> 4c80e9236961
Step 14/15 : ENTRYPOINT [ "docker-entrypoint.sh" ]
\ ---> Running in 27e64d0f518e
- ---> 9297f46691a5
Step 15/15 : CMD ["bentoml", "serve-gunicorn", "/bento"]
 ---> Running in eef46c6f5c73
/ ---> 6b11f7a9f3d1
Successfully built 6b11f7a9f3d1
Successfully tagged titanicsurvivalpredictionservice:20200922123336_3D25BB
Finished building titanicsurvivalpredictionservice:20200922123336_3D25BB from TitanicSurvivalPredictionService:latest
In [15]:
!docker run --rm -p 5000:5000 titanicsurvivalpredictionservice:20200922123336_3D25BB
[2020-09-22 19:37:28,365] INFO - Starting BentoML API server in production mode..
[2020-09-22 19:37:28,788] INFO - get_gunicorn_num_of_workers: 3, calculated by cpu count
[2020-09-22 19:37:28 +0000] [1] [INFO] Starting gunicorn 20.0.4
[2020-09-22 19:37:28 +0000] [1] [INFO] Listening at: http://0.0.0.0:5000 (1)
[2020-09-22 19:37:28 +0000] [1] [INFO] Using worker: sync
[2020-09-22 19:37:28 +0000] [11] [INFO] Booting worker with pid: 11
[2020-09-22 19:37:28 +0000] [12] [INFO] Booting worker with pid: 12
[2020-09-22 19:37:28 +0000] [13] [INFO] Booting worker with pid: 13
[2020-09-22 19:37:29,051] WARNING - Using BentoML not from official PyPI release. In order to find the same version of BentoML when deploying your BentoService, you must set the 'core/bentoml_deploy_version' config to a http/git location of your BentoML fork, e.g.: 'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
[2020-09-22 19:37:29,071] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015
[2020-09-22 19:37:29,072] WARNING - Saved BentoService Python version mismatch: loading BentoService bundle created with Python version 3.7.3, but current environment version is 3.7.6.
[2020-09-22 19:37:29,092] WARNING - Using BentoML not from official PyPI release. In order to find the same version of BentoML when deploying your BentoService, you must set the 'core/bentoml_deploy_version' config to a http/git location of your BentoML fork, e.g.: 'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
[2020-09-22 19:37:29,119] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015
[2020-09-22 19:37:29,121] WARNING - Saved BentoService Python version mismatch: loading BentoService bundle created with Python version 3.7.3, but current environment version is 3.7.6.
[2020-09-22 19:37:29,173] WARNING - Using BentoML not from official PyPI release. In order to find the same version of BentoML when deploying your BentoService, you must set the 'core/bentoml_deploy_version' config to a http/git location of your BentoML fork, e.g.: 'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
[2020-09-22 19:37:29,216] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015
[2020-09-22 19:37:29,217] WARNING - Saved BentoService Python version mismatch: loading BentoService bundle created with Python version 3.7.3, but current environment version is 3.7.6.
[2020-09-22 19:37:30,169] WARNING - pip package requirement lightgbm already exist
[2020-09-22 19:37:30,193] WARNING - pip package requirement lightgbm already exist
[2020-09-22 19:37:30,237] WARNING - pip package requirement lightgbm already exist
^C
[2020-09-22 19:37:31 +0000] [1] [INFO] Handling signal: int
[2020-09-22 19:37:31 +0000] [12] [INFO] Worker exiting (pid: 12)
[2020-09-22 19:37:31 +0000] [13] [INFO] Worker exiting (pid: 13)
[2020-09-22 19:37:31 +0000] [11] [INFO] Worker exiting (pid: 11)

Load saved BentoService for serving

In [16]:
import bentoml

bento_model = bentoml.load(saved_path)

result = bento_model.predict(test_df)
test_df['pred'] = result
test_df[['Pclass', 'Age', 'Fare', 'SibSp', 'Parch','pred']].iloc[10:].head(2)
[2020-09-22 12:37:36,142] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015
[2020-09-22 12:37:36,145] WARNING - Module `lightbgm_titanic_bento_service` already loaded, using existing imported module.
[2020-09-22 12:37:36,176] WARNING - pip package requirement pandas already exist
[2020-09-22 12:37:36,178] WARNING - pip package requirement lightgbm already exist
Out[16]:
Pclass Age Fare SibSp Parch pred
10 3 NaN 7.8958 0 0 0.052353
11 1 46.0 26.0000 0 0 0.308877

Launch inference job from CLI

BentoML cli supports loading and running a packaged model from CLI. With the DataframeInput adapter, the CLI command supports reading input Dataframe data from CLI argument or local csv or json files:

In [17]:
!bentoml run TitanicSurvivalPredictionService:latest predict \
--input '{"PassengerId":{"3":895},"Pclass":{"3":3},"Name":{"3":"Wirz, Mr. Albert"},"Sex":{"3":"male"},"Age":{"3":27.0},"SibSp":{"3":0},"Parch":{"3":0},"Ticket":{"3":"315154"},"Fare":{"3":8.6625},"Cabin":{"3":null},"Embarked":{"3":"S"},"pred":{"3":0.5045963287}}'
[2020-09-22 12:37:51,453] INFO - Getting latest version TitanicSurvivalPredictionService:20200922123336_3D25BB
[2020-09-22 12:37:51,493] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.
[2020-09-22 12:37:51,508] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015
[2020-09-22 12:37:53,412] INFO - Using default docker base image: `None` specified inBentoML config file or env var. User must make sure that the docker base image either has Python 3.7 or conda installed.
[2020-09-22 12:37:53,439] WARNING - pip package requirement lightgbm already exist
[2020-09-22 12:37:57,015] INFO - {'service_name': 'TitanicSurvivalPredictionService', 'service_version': '20200922123336_3D25BB', 'api': 'predict', 'task': {'data': {}, 'task_id': '60b02429-936a-4aa0-b15a-8cd9eb9c464f', 'batch': 1, 'cli_args': ('--input', '{"PassengerId":{"3":895},"Pclass":{"3":3},"Name":{"3":"Wirz, Mr. Albert"},"Sex":{"3":"male"},"Age":{"3":27.0},"SibSp":{"3":0},"Parch":{"3":0},"Ticket":{"3":"315154"},"Fare":{"3":8.6625},"Cabin":{"3":null},"Embarked":{"3":"S"},"pred":{"3":0.5045963287}}')}, 'result': {'data': '[0.5045963286563061]', 'http_status': 200, 'http_headers': (('Content-Type', 'application/json'),)}, 'request_id': '60b02429-936a-4aa0-b15a-8cd9eb9c464f'}
[0.5045963286563061]

Deployment Options

If you are at a small team with limited engineering or DevOps resources, try out automated deployment with BentoML CLI, currently supporting AWS Lambda, AWS SageMaker, and Azure Functions:

If the cloud platform you are working with is not on the list above, try out these step-by-step guide on manually deploying BentoML packaged model to cloud platforms:

Lastly, if you have a DevOps or ML Engineering team who's operating a Kubernetes or OpenShift cluster, use the following guides as references for implementating your deployment strategy:

In [ ]: