1. gallery
  2. xgboost
  3. titanic-survival-prediction

BentoML Demo: Titanic Survival Prediction with XGBoost

BentoML makes moving trained ML models to production easy:

  • Package models trained with any ML framework and reproduce them for model serving in production
  • Deploy anywhere for online API serving or offline batch serving
  • High-Performance API model server with adaptive micro-batching support
  • Central hub for managing models and deployment process via Web UI and APIs
  • Modular and flexible design making it adaptable to your infrastrcuture

BentoML is a framework for serving, managing, and deploying machine learning models. It is aiming to bridge the gap between Data Science and DevOps, and enable teams to deliver prediction services in a fast, repeatable, and scalable way.

Before reading this example project, be sure to check out the Getting started guide to learn about the basic concepts in BentoML.

This notebook demonstrates use BentoML to serve a model trained with the XGBoost framework, specifically using the Titanic Survival dataset.

Let's get started! Impression

In [1]:
%reload_ext autoreload
%autoreload 2
%matplotlib inline

import warnings
warnings.filterwarnings("ignore")
In [2]:
!pip install -q --upgrade xgboost==0.90 numpy==1.18.5 pandas==1.0.4 bentoml
In [2]:
import pandas as pd
import numpy as np
import xgboost as xgb
import bentoml

Prepare Dataset

download dataset from https://www.kaggle.com/c/titanic/data

In [3]:
!mkdir data
!curl https://raw.githubusercontent.com/agconti/kaggle-titanic/master/data/train.csv -o ./data/train.csv
!curl https://raw.githubusercontent.com/agconti/kaggle-titanic/master/data/test.csv -o ./data/test.csv

mkdir: data: File exists
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 60302  100 60302    0     0   133k      0 --:--:-- --:--:-- --:--:--  133k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 28210  100 28210    0     0  62273      0 --:--:-- --:--:-- --:--:-- 62273
In [4]:
train = pd.read_csv("./data/train.csv")
test  = pd.read_csv("./data/test.csv")
X_y_train = xgb.DMatrix(data=train[['Pclass', 'Age', 'Fare', 'SibSp', 'Parch']], label= train['Survived'])
X_test    = xgb.DMatrix(data=test[['Pclass', 'Age', 'Fare', 'SibSp', 'Parch']])
In [5]:
train[['Pclass', 'Age', 'Fare', 'SibSp', 'Parch', 'Survived']].head()
Out[5]:
Pclass Age Fare SibSp Parch Survived
0 3 22.0 7.2500 1 0 0
1 1 38.0 71.2833 1 0 1
2 3 26.0 7.9250 0 0 1
3 1 35.0 53.1000 1 0 1
4 3 35.0 8.0500 0 0 0

Model Training

In [6]:
params = {
          'base_score': np.mean(train['Survived']),
          'eta':  0.1,
          'max_depth': 3,
          'gamma' :3,
          'objective'   :'reg:linear',
          'eval_metric' :'mae'
         }
model = xgb.train(params=params, 
                  dtrain=X_y_train, 
                  num_boost_round=3)

[15:41:43] WARNING: /Users/travis/build/dmlc/xgboost/src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[15:41:43] WARNING: /Users/travis/build/dmlc/xgboost/src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
In [7]:
y_test =  model.predict(X_test)
test['pred'] = y_test
test[['Pclass', 'Age', 'Fare', 'SibSp', 'Parch','pred']].iloc[10:].head(2)
Out[7]:
Pclass Age Fare SibSp Parch pred
10 3 NaN 7.8958 0 0 0.341580
11 1 46.0 26.0000 0 0 0.413966

Create BentoService for model serving

In [8]:
%%writefile xgboost_titanic_bento_service.py

import xgboost as xgb

import bentoml
from bentoml.frameworks.xgboost import XgboostModelArtifact
from bentoml.adapters import DataframeInput

@bentoml.env(infer_pip_packages=True)
@bentoml.artifacts([XgboostModelArtifact('model')])
class TitanicSurvivalPredictionXgBoost(bentoml.BentoService):
    
    @bentoml.api(input=DataframeInput(), batch=True)
    def predict(self, df):
        data = xgb.DMatrix(data=df[['Pclass', 'Age', 'Fare', 'SibSp', 'Parch']])
        return self.artifacts.model.predict(data)

Overwriting xgboost_titanic_bento_service.py

Create BentoService saved bundle

In [9]:
# 1) import the custom BentoService defined above
from xgboost_titanic_bento_service import TitanicSurvivalPredictionXgBoost

# 2) `pack` it with required artifacts
bento_service = TitanicSurvivalPredictionXgBoost()
bento_service.pack('model', model)

# 3) save your BentoSerivce
saved_path = bento_service.save()

[2020-09-22 15:42:22,025] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.
[2020-09-22 15:42:22,375] INFO - Using default docker base image: `None` specified inBentoML config file or env var. User must make sure that the docker base image either has Python 3.7 or conda installed.
[2020-09-22 15:42:23,380] INFO - Detected non-PyPI-released BentoML installed, copying local BentoML modulefiles to target saved bundle path..

warning: no previously-included files matching '*~' found anywhere in distribution
warning: no previously-included files matching '*.pyo' found anywhere in distribution
warning: no previously-included files matching '.git' found anywhere in distribution
warning: no previously-included files matching '.ipynb_checkpoints' found anywhere in distribution
warning: no previously-included files matching '__pycache__' found anywhere in distribution
no previously-included directories found matching 'e2e_tests'
no previously-included directories found matching 'tests'
no previously-included directories found matching 'benchmark'

UPDATING BentoML-0.9.0rc0+3.gcebf2015/bentoml/_version.py
set BentoML-0.9.0rc0+3.gcebf2015/bentoml/_version.py to '0.9.0.pre+3.gcebf2015'
[2020-09-22 15:42:27,403] INFO - BentoService bundle 'TitanicSurvivalPredictionXgBoost:20200922154223_4ACDA4' saved to: /Users/bozhaoyu/bentoml/repository/TitanicSurvivalPredictionXgBoost/20200922154223_4ACDA4

REST API Model Serving

To start a REST API model server with the BentoService saved above, use the bentoml serve command:

In [10]:
!bentoml serve TitanicSurvivalPredictionXgBoost:latest

[2020-09-22 15:43:00,146] INFO - Getting latest version TitanicSurvivalPredictionXgBoost:20200922154223_4ACDA4
[2020-09-22 15:43:00,146] INFO - Starting BentoML API server in development mode..
[2020-09-22 15:43:00,389] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.
[2020-09-22 15:43:00,404] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015
[2020-09-22 15:43:01,341] INFO - Using default docker base image: `None` specified inBentoML config file or env var. User must make sure that the docker base image either has Python 3.7 or conda installed.
[15:43:01] WARNING: /Users/travis/build/dmlc/xgboost/src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
 * Serving Flask app "TitanicSurvivalPredictionXgBoost" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
[2020-09-22 15:43:10,246] INFO - {'service_name': 'TitanicSurvivalPredictionXgBoost', 'service_version': '20200922154223_4ACDA4', 'api': 'predict', 'task': {'data': {}, 'task_id': '72674164-f525-4914-8df7-d351ebba8079', 'batch': 1, 'http_headers': (('Host', 'localhost:5000'), ('User-Agent', 'curl/7.65.3'), ('Accept', '*/*'), ('Content-Type', 'application/json'), ('Content-Length', '63'))}, 'result': {'data': '[0.469721257686615]', 'http_status': 200, 'http_headers': (('Content-Type', 'application/json'),)}, 'request_id': '72674164-f525-4914-8df7-d351ebba8079'}
127.0.0.1 - - [22/Sep/2020 15:43:10] "POST /predict HTTP/1.1" 200 -
WARNING: Logging before flag parsing goes to stderr.
I0922 15:43:10.249207 4654042560 _internal.py:122] 127.0.0.1 - - [22/Sep/2020 15:43:10] "POST /predict HTTP/1.1" 200 -
^C

If you are running this notebook from Google Colab, you can start the dev server with --run-with-ngrok option, to gain acccess to the API endpoint via a public endpoint managed by ngrok:

In [ ]:
!bentoml serve TitanicSurvivalPredictionXgBoost:latest --run-with-ngrok

Open http://127.0.0.1:5000 to see more information about the REST APIs server in your browser.

Send prediction requeset to the REST API server

Run the following curl command to send request data to REST API server and get a prediction result:

curl -i \
--header "Content-Type: application/json" \
--request POST \
--data '[{"Pclass": 1, "Age": 30, "Fare": 200, "SibSp": 1, "Parch": 0}]' \
localhost:5000/predict

Containerize model server with Docker

One common way of distributing this model API server for production deployment, is via Docker containers. And BentoML provides a convenient way to do that.

Note that docker is not available in Google Colab. You will need to download and run this notebook locally to try out this containerization with docker feature.

If you already have docker configured, simply run the follow command to product a docker container serving the IrisClassifier prediction service created above:

In [11]:
!bentoml containerize TitanicSurvivalPredictionXgBoost:latest

[2020-09-22 15:43:25,886] INFO - Getting latest version TitanicSurvivalPredictionXgBoost:20200922154223_4ACDA4
Found Bento: /Users/bozhaoyu/bentoml/repository/TitanicSurvivalPredictionXgBoost/20200922154223_4ACDA4
[2020-09-22 15:43:25,932] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.
[2020-09-22 15:43:25,958] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015
Tag not specified, using tag parsed from BentoService: 'titanicsurvivalpredictionxgboost:20200922154223_4ACDA4'
Building Docker image titanicsurvivalpredictionxgboost:20200922154223_4ACDA4 from TitanicSurvivalPredictionXgBoost:latest 
-we in here
processed docker file
(None, None)
root in create archive /Users/bozhaoyu/bentoml/repository/TitanicSurvivalPredictionXgBoost/20200922154223_4ACDA4 ['Dockerfile', 'MANIFEST.in', 'README.md', 'TitanicSurvivalPredictionXgBoost', 'TitanicSurvivalPredictionXgBoost/__init__.py', 'TitanicSurvivalPredictionXgBoost/__pycache__', 'TitanicSurvivalPredictionXgBoost/__pycache__/xgboost_titanic_bento_service.cpython-37.pyc', 'TitanicSurvivalPredictionXgBoost/artifacts', 'TitanicSurvivalPredictionXgBoost/artifacts/__init__.py', 'TitanicSurvivalPredictionXgBoost/artifacts/model.model', 'TitanicSurvivalPredictionXgBoost/bentoml.yml', 'TitanicSurvivalPredictionXgBoost/xgboost_titanic_bento_service.py', 'bentoml-init.sh', 'bentoml.yml', 'bundled_pip_dependencies', 'bundled_pip_dependencies/BentoML-0.9.0rc0+3.gcebf2015.tar.gz', 'docker-entrypoint.sh', 'environment.yml', 'python_version', 'requirements.txt', 'setup.py']
about to build
about to upgrade params
check each param and update
if use config proxy
if buildargs
if shmsize
if labels
if cache from
if target
if network_mode
if squash
if extra hosts is not None
if platform is not None
if isolcation is not None
if context is not None
setting auth {'Content-Type': 'application/tar'}
-docker build <tempfile._TemporaryFileWrapper object at 0x7fa11a5b3da0> {'t': 'titanicsurvivalpredictionxgboost:20200922154223_4ACDA4', 'remote': None, 'q': False, 'nocache': False, 'rm': False, 'forcerm': False, 'pull': False, 'dockerfile': (None, None)}
/docker response <Response [200]>
context closes
print responses
Step 1/15 : FROM bentoml/model-server:0.9.0.pre
 ---> a25066aa8b0e
Step 2/15 : ARG EXTRA_PIP_INSTALL_ARGS=
 ---> Using cache
 ---> fc6e47d06522
Step 3/15 : ENV EXTRA_PIP_INSTALL_ARGS $EXTRA_PIP_INSTALL_ARGS
 ---> Using cache
 ---> db8172e98571
Step 4/15 : COPY environment.yml requirements.txt setup.sh* bentoml-init.sh python_version* /bento/
| ---> a10efa4b16c9
Step 5/15 : WORKDIR /bento
\ ---> Running in 5f8d65af7025
/ ---> beef999ddff3
Step 6/15 : RUN chmod +x /bento/bentoml-init.sh
 ---> Running in f3ca77c6fe53
/ ---> 5ef5b7bac8f1
Step 7/15 : RUN if [ -f /bento/bentoml-init.sh ]; then bash -c /bento/bentoml-init.sh; fi
 ---> Running in da374f23bb44
-+++ dirname /bento/bentoml-init.sh

++ cd /bento

++ pwd -P

+ SAVED_BUNDLE_PATH=/bento
+ cd /bento
+ '[' -f ./setup.sh ']'
+ '[' -f ./python_version ']'

++ cat ./python_version

/+ PY_VERSION_SAVED=3.7.3
+ DESIRED_PY_VERSION=3.7

++ python -c 'import sys; print(f"{sys.version_info.major}.{sys.version_info.minor}")'

Python Version in docker base image 3.7 matches requirement python=3.7. Skipping.
Updating conda base environment with environment.yml
+ CURRENT_PY_VERSION=3.7
+ [[ 3.7 == \3\.\7 ]]
+ echo 'Python Version in docker base image 3.7 matches requirement python=3.7. Skipping.'
+ command -v conda
+ echo 'Updating conda base environment with environment.yml'
+ conda env update -n base -f ./environment.yml

-Collecting package metadata (repodata.json): ...working... 
/done
Solving environment: ...working... 
\done
-
Downloading and Extracting Packages
python_abi-3.7       | 4 KB      |            |   0% 
python_abi-3.7       | 4 KB      | ########## | 100% 
python_abi-3.7       | 4 KB      | ########## | 100% 

pip-20.2.3           | 1.1 MB    |            |   0% 
pip-20.2.3           | 1.1 MB    | ##3        |  23% 
pip-20.2.3           | 1.1 MB    | ########## | 100% 
pip-20.2.3           | 1.1 MB    | ########## | 100% 

openssl-1.1.1h       | 2.1 MB    |            |   0% 
openssl-1.1.1h       | 2.1 MB    | ####3      |  43% 
openssl-1.1.1h       | 2.1 MB    | ########## | 100% 
openssl-1.1.1h       | 2.1 MB    | ########## | 100% 

ca-certificates-2020 | 145 KB    |            |   0% 
ca-certificates-2020 | 145 KB    | ########## | 100% 

cffi-1.14.3          | 223 KB    |            |   0% 
cffi-1.14.3          | 223 KB    | ########## | 100% 
cffi-1.14.3          | 223 KB    | ########## | 100% 

certifi-2020.6.20    | 151 KB    |            |   0% 
certifi-2020.6.20    | 151 KB    | ########## | 100% 
certifi-2020.6.20    | 151 KB    | ########## | 100% 

libffi-3.2.1         | 47 KB     |            |   0% 
libffi-3.2.1         | 47 KB     | ########## | 100% 

Preparing transaction: ...working... 
done
Verifying transaction: ...working... 
-done
Executing transaction: ...working... 
\done
/#
# To activate this environment, use
#
#     $ conda activate base
#
# To deactivate an active environment, use
#
#     $ conda deactivate
|+ pip install -r ./requirements.txt --no-cache-dir

\Requirement already satisfied: bentoml==0.9.0.pre in /opt/conda/lib/python3.7/site-packages (from -r ./requirements.txt (line 1)) (0.9.0rc0)
/Collecting pandas==0.24.2
|  Downloading pandas-0.24.2-cp37-cp37m-manylinux1_x86_64.whl (10.1 MB)
|Collecting xgboost==1.2.0
  Downloading xgboost-1.2.0-py3-none-manylinux2010_x86_64.whl (148.9 MB)
-Requirement already satisfied: humanfriendly in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (8.2)
Requirement already satisfied: python-json-logger in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.1.11)
Requirement already satisfied: flask in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.1.2)
Requirement already satisfied: configparser in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (5.0.0)
Requirement already satisfied: numpy in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.19.2)
Requirement already satisfied: prometheus-client in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.8.0)
Requirement already satisfied: sqlalchemy>=1.3.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.3.19)
Requirement already satisfied: tabulate in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.8.7)
Requirement already satisfied: cerberus in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.3.2)
Requirement already satisfied: sqlalchemy-utils<0.36.8 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.36.7)
/Requirement already satisfied: python-dateutil<3.0.0,>=2.7.3 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (2.8.1)
Requirement already satisfied: alembic in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.4.3)
Requirement already satisfied: multidict in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (4.7.6)
Requirement already satisfied: grpcio<=1.27.2 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.27.2)
Requirement already satisfied: gunicorn in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (20.0.4)
Requirement already satisfied: ruamel.yaml>=0.15.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.15.87)
Requirement already satisfied: certifi in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (2020.6.20)
Requirement already satisfied: py-zipkin in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.20.0)
Requirement already satisfied: protobuf>=3.6.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (3.13.0)
Requirement already satisfied: docker in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (4.3.1)
|Requirement already satisfied: aiohttp in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (3.6.2)
Requirement already satisfied: boto3 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.15.2)
Requirement already satisfied: psutil in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (5.7.2)
Requirement already satisfied: packaging in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (20.4)
Requirement already satisfied: click>=7.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (7.1.2)
Requirement already satisfied: requests in /opt/conda/lib/python3.7/site-packages (from bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (2.24.0)
/Collecting pytz>=2011k
  Downloading pytz-2020.1-py2.py3-none-any.whl (510 kB)
-Collecting scipy
/  Downloading scipy-1.5.2-cp37-cp37m-manylinux1_x86_64.whl (25.9 MB)
/Requirement already satisfied: Werkzeug>=0.15 in /opt/conda/lib/python3.7/site-packages (from flask->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.0.1)
Requirement already satisfied: itsdangerous>=0.24 in /opt/conda/lib/python3.7/site-packages (from flask->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.1.0)
Requirement already satisfied: Jinja2>=2.10.1 in /opt/conda/lib/python3.7/site-packages (from flask->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (2.11.2)
Requirement already satisfied: setuptools in /opt/conda/lib/python3.7/site-packages (from cerberus->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (49.6.0.post20200814)
Requirement already satisfied: six in /opt/conda/lib/python3.7/site-packages (from sqlalchemy-utils<0.36.8->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.15.0)
Requirement already satisfied: Mako in /opt/conda/lib/python3.7/site-packages (from alembic->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.1.3)
Requirement already satisfied: python-editor>=0.3 in /opt/conda/lib/python3.7/site-packages (from alembic->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.0.4)
Requirement already satisfied: thriftpy2>=0.4.0 in /opt/conda/lib/python3.7/site-packages (from py-zipkin->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.4.11)
Requirement already satisfied: websocket-client>=0.32.0 in /opt/conda/lib/python3.7/site-packages (from docker->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.57.0)
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.5.1)
Requirement already satisfied: chardet<4.0,>=2.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (3.0.4)
Requirement already satisfied: async-timeout<4.0,>=3.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (3.0.1)
Requirement already satisfied: attrs>=17.3.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (20.2.0)
|Requirement already satisfied: s3transfer<0.4.0,>=0.3.0 in /opt/conda/lib/python3.7/site-packages (from boto3->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.3.3)
Requirement already satisfied: botocore<1.19.0,>=1.18.2 in /opt/conda/lib/python3.7/site-packages (from boto3->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.18.2)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /opt/conda/lib/python3.7/site-packages (from boto3->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (0.10.0)
Requirement already satisfied: pyparsing>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from packaging->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (2.4.7)
Requirement already satisfied: idna<3,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (2.10)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.25.10)
Requirement already satisfied: MarkupSafe>=0.23 in /opt/conda/lib/python3.7/site-packages (from Jinja2>=2.10.1->flask->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (1.1.1)
Requirement already satisfied: ply<4.0,>=3.4 in /opt/conda/lib/python3.7/site-packages (from thriftpy2>=0.4.0->py-zipkin->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (3.11)
Requirement already satisfied: typing-extensions>=3.7.4; python_version < "3.8" in /opt/conda/lib/python3.7/site-packages (from yarl<2.0,>=1.0->aiohttp->bentoml==0.9.0.pre->-r ./requirements.txt (line 1)) (3.7.4.3)
\Installing collected packages: pytz, pandas, scipy, xgboost
/Successfully installed pandas-0.24.2 pytz-2020.1 scipy-1.5.2 xgboost-1.2.0
\ ---> aa4db686ba89
Step 8/15 : COPY . /bento
| ---> 4bafca014215
Step 9/15 : RUN if [ -d /bento/bundled_pip_dependencies ]; then pip install -U bundled_pip_dependencies/* ;fi
 ---> Running in 6d5fa028c021
-Processing ./bundled_pip_dependencies/BentoML-0.9.0rc0+3.gcebf2015.tar.gz
-  Installing build dependencies: started
\  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
-  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
|    Preparing wheel metadata: finished with status 'done'
-Requirement already satisfied, skipping upgrade: click>=7.0 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (7.1.2)
Requirement already satisfied, skipping upgrade: docker in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (4.3.1)
Requirement already satisfied, skipping upgrade: multidict in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (4.7.6)
Requirement already satisfied, skipping upgrade: sqlalchemy-utils<0.36.8 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (0.36.7)
/Requirement already satisfied, skipping upgrade: requests in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (2.24.0)
Requirement already satisfied, skipping upgrade: flask in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (1.1.2)
Requirement already satisfied, skipping upgrade: grpcio<=1.27.2 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (1.27.2)
Requirement already satisfied, skipping upgrade: configparser in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (5.0.0)
Requirement already satisfied, skipping upgrade: py-zipkin in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (0.20.0)
Requirement already satisfied, skipping upgrade: alembic in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (1.4.3)
Requirement already satisfied, skipping upgrade: sqlalchemy>=1.3.0 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (1.3.19)
|Requirement already satisfied, skipping upgrade: python-dateutil<3.0.0,>=2.7.3 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (2.8.1)
Requirement already satisfied, skipping upgrade: packaging in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (20.4)
Requirement already satisfied, skipping upgrade: protobuf>=3.6.0 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (3.13.0)
Requirement already satisfied, skipping upgrade: tabulate in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (0.8.7)
Requirement already satisfied, skipping upgrade: gunicorn in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (20.0.4)
Requirement already satisfied, skipping upgrade: psutil in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (5.7.2)
Requirement already satisfied, skipping upgrade: humanfriendly in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (8.2)
Requirement already satisfied, skipping upgrade: ruamel.yaml>=0.15.0 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (0.15.87)
Requirement already satisfied, skipping upgrade: numpy in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (1.19.2)
Requirement already satisfied, skipping upgrade: python-json-logger in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (0.1.11)
Requirement already satisfied, skipping upgrade: aiohttp in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (3.6.2)
Requirement already satisfied, skipping upgrade: certifi in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (2020.6.20)
Requirement already satisfied, skipping upgrade: cerberus in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (1.3.2)
Requirement already satisfied, skipping upgrade: boto3 in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (1.15.2)
Requirement already satisfied, skipping upgrade: prometheus-client in /opt/conda/lib/python3.7/site-packages (from BentoML==0.9.0rc0+3.gcebf2015) (0.8.0)
Requirement already satisfied, skipping upgrade: websocket-client>=0.32.0 in /opt/conda/lib/python3.7/site-packages (from docker->BentoML==0.9.0rc0+3.gcebf2015) (0.57.0)
Requirement already satisfied, skipping upgrade: six>=1.4.0 in /opt/conda/lib/python3.7/site-packages (from docker->BentoML==0.9.0rc0+3.gcebf2015) (1.15.0)
Requirement already satisfied, skipping upgrade: idna<3,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests->BentoML==0.9.0rc0+3.gcebf2015) (2.10)
Requirement already satisfied, skipping upgrade: chardet<4,>=3.0.2 in /opt/conda/lib/python3.7/site-packages (from requests->BentoML==0.9.0rc0+3.gcebf2015) (3.0.4)
\Requirement already satisfied, skipping upgrade: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests->BentoML==0.9.0rc0+3.gcebf2015) (1.25.10)
Requirement already satisfied, skipping upgrade: itsdangerous>=0.24 in /opt/conda/lib/python3.7/site-packages (from flask->BentoML==0.9.0rc0+3.gcebf2015) (1.1.0)
Requirement already satisfied, skipping upgrade: Jinja2>=2.10.1 in /opt/conda/lib/python3.7/site-packages (from flask->BentoML==0.9.0rc0+3.gcebf2015) (2.11.2)
Requirement already satisfied, skipping upgrade: Werkzeug>=0.15 in /opt/conda/lib/python3.7/site-packages (from flask->BentoML==0.9.0rc0+3.gcebf2015) (1.0.1)
Requirement already satisfied, skipping upgrade: thriftpy2>=0.4.0 in /opt/conda/lib/python3.7/site-packages (from py-zipkin->BentoML==0.9.0rc0+3.gcebf2015) (0.4.11)
Requirement already satisfied, skipping upgrade: python-editor>=0.3 in /opt/conda/lib/python3.7/site-packages (from alembic->BentoML==0.9.0rc0+3.gcebf2015) (1.0.4)
Requirement already satisfied, skipping upgrade: Mako in /opt/conda/lib/python3.7/site-packages (from alembic->BentoML==0.9.0rc0+3.gcebf2015) (1.1.3)
Requirement already satisfied, skipping upgrade: pyparsing>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from packaging->BentoML==0.9.0rc0+3.gcebf2015) (2.4.7)
Requirement already satisfied, skipping upgrade: setuptools in /opt/conda/lib/python3.7/site-packages (from protobuf>=3.6.0->BentoML==0.9.0rc0+3.gcebf2015) (49.6.0.post20200814)
Requirement already satisfied, skipping upgrade: async-timeout<4.0,>=3.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->BentoML==0.9.0rc0+3.gcebf2015) (3.0.1)
Requirement already satisfied, skipping upgrade: attrs>=17.3.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->BentoML==0.9.0rc0+3.gcebf2015) (20.2.0)
Requirement already satisfied, skipping upgrade: yarl<2.0,>=1.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->BentoML==0.9.0rc0+3.gcebf2015) (1.5.1)
Requirement already satisfied, skipping upgrade: botocore<1.19.0,>=1.18.2 in /opt/conda/lib/python3.7/site-packages (from boto3->BentoML==0.9.0rc0+3.gcebf2015) (1.18.2)
-Requirement already satisfied, skipping upgrade: jmespath<1.0.0,>=0.7.1 in /opt/conda/lib/python3.7/site-packages (from boto3->BentoML==0.9.0rc0+3.gcebf2015) (0.10.0)
Requirement already satisfied, skipping upgrade: s3transfer<0.4.0,>=0.3.0 in /opt/conda/lib/python3.7/site-packages (from boto3->BentoML==0.9.0rc0+3.gcebf2015) (0.3.3)
Requirement already satisfied, skipping upgrade: MarkupSafe>=0.23 in /opt/conda/lib/python3.7/site-packages (from Jinja2>=2.10.1->flask->BentoML==0.9.0rc0+3.gcebf2015) (1.1.1)
Requirement already satisfied, skipping upgrade: ply<4.0,>=3.4 in /opt/conda/lib/python3.7/site-packages (from thriftpy2>=0.4.0->py-zipkin->BentoML==0.9.0rc0+3.gcebf2015) (3.11)
Requirement already satisfied, skipping upgrade: typing-extensions>=3.7.4; python_version < "3.8" in /opt/conda/lib/python3.7/site-packages (from yarl<2.0,>=1.0->aiohttp->BentoML==0.9.0rc0+3.gcebf2015) (3.7.4.3)
Building wheels for collected packages: BentoML
  Building wheel for BentoML (PEP 517): started
/  Building wheel for BentoML (PEP 517): finished with status 'done'
  Created wheel for BentoML: filename=BentoML-0.9.0rc0+3.gcebf2015-py3-none-any.whl size=3064091 sha256=a49ace9338ca06346a0ec6749888d021774ab66106b4617a2935d82006d25f23
  Stored in directory: /root/.cache/pip/wheels/a0/45/41/62152db705af4ff47e7a3d6abf6247986eef4aa1b94a58d3b9
Successfully built BentoML
/Installing collected packages: BentoML
  Attempting uninstall: BentoML
    Found existing installation: BentoML 0.9.0rc0
\    Uninstalling BentoML-0.9.0rc0:
/      Successfully uninstalled BentoML-0.9.0rc0
\Successfully installed BentoML-0.9.0rc0+3.gcebf2015
 ---> 6cd0ae85c1b2
-Step 10/15 : ENV PORT 5000
 ---> Running in cd6452eb4170
/ ---> 0487d34f357e
Step 11/15 : EXPOSE $PORT
 ---> Running in e90e3a5071a6
\ ---> 9ec18ef48e95
Step 12/15 : COPY docker-entrypoint.sh /usr/local/bin/
- ---> 7de3d800cd69
Step 13/15 : RUN chmod +x /usr/local/bin/docker-entrypoint.sh
/ ---> Running in 38911bbbc5c6
| ---> 2b4f7f4efcaf
Step 14/15 : ENTRYPOINT [ "docker-entrypoint.sh" ]
\ ---> Running in 1aa5f46c04ce
- ---> 156b1f48734c
Step 15/15 : CMD ["bentoml", "serve-gunicorn", "/bento"]
 ---> Running in 3b5142efcdba
/ ---> 19d4b648b2a6
Successfully built 19d4b648b2a6
Successfully tagged titanicsurvivalpredictionxgboost:20200922154223_4ACDA4
Finished building titanicsurvivalpredictionxgboost:20200922154223_4ACDA4 from TitanicSurvivalPredictionXgBoost:latest

Next, you can docker push the image to your choice of registry for deployment, or run it locally for development and testing:

In [12]:
!docker run -p 5000:5000 titanicsurvivalpredictionxgboost:20200922154223_4ACDA4

[2020-09-22 22:47:14,344] INFO - Starting BentoML API server in production mode..
[2020-09-22 22:47:14,742] INFO - get_gunicorn_num_of_workers: 3, calculated by cpu count
[2020-09-22 22:47:14 +0000] [1] [INFO] Starting gunicorn 20.0.4
[2020-09-22 22:47:14 +0000] [1] [INFO] Listening at: http://0.0.0.0:5000 (1)
[2020-09-22 22:47:14 +0000] [1] [INFO] Using worker: sync
[2020-09-22 22:47:14 +0000] [11] [INFO] Booting worker with pid: 11
[2020-09-22 22:47:14 +0000] [12] [INFO] Booting worker with pid: 12
[2020-09-22 22:47:14 +0000] [13] [INFO] Booting worker with pid: 13
[2020-09-22 22:47:15,001] WARNING - Using BentoML not from official PyPI release. In order to find the same version of BentoML when deploying your BentoService, you must set the 'core/bentoml_deploy_version' config to a http/git location of your BentoML fork, e.g.: 'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
[2020-09-22 22:47:15,023] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015
[2020-09-22 22:47:15,024] WARNING - Saved BentoService Python version mismatch: loading BentoService bundle created with Python version 3.7.3, but current environment version is 3.7.6.
[2020-09-22 22:47:15,054] WARNING - Using BentoML not from official PyPI release. In order to find the same version of BentoML when deploying your BentoService, you must set the 'core/bentoml_deploy_version' config to a http/git location of your BentoML fork, e.g.: 'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
[2020-09-22 22:47:15,076] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015
[2020-09-22 22:47:15,076] WARNING - Saved BentoService Python version mismatch: loading BentoService bundle created with Python version 3.7.3, but current environment version is 3.7.6.
[2020-09-22 22:47:15,198] WARNING - Using BentoML not from official PyPI release. In order to find the same version of BentoML when deploying your BentoService, you must set the 'core/bentoml_deploy_version' config to a http/git location of your BentoML fork, e.g.: 'bentoml_deploy_version = git+https://github.com/{username}/[email protected]{branch}'
[2020-09-22 22:47:15,219] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015
[2020-09-22 22:47:15,219] WARNING - Saved BentoService Python version mismatch: loading BentoService bundle created with Python version 3.7.3, but current environment version is 3.7.6.
^C
[2020-09-22 22:47:18 +0000] [1] [INFO] Handling signal: int
[2020-09-22 22:47:18 +0000] [12] [INFO] Worker exiting (pid: 12)
[2020-09-22 22:47:18 +0000] [11] [INFO] Worker exiting (pid: 11)
[2020-09-22 22:47:18 +0000] [13] [INFO] Worker exiting (pid: 13)

Load saved BentoService

bentoml.load is the API for loading a BentoML packaged model in python:

In [13]:
import bentoml

loaded_svc = bentoml.load(saved_path)

result = loaded_svc.predict(test)
test['pred'] = result
test[['Pclass', 'Age', 'Fare', 'SibSp', 'Parch','pred']]

[2020-09-22 15:47:22,359] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015
[2020-09-22 15:47:22,360] WARNING - Module `xgboost_titanic_bento_service` already loaded, using existing imported module.
[15:47:22] WARNING: /Users/travis/build/dmlc/xgboost/src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[2020-09-22 15:47:22,363] WARNING - pip package requirement pandas already exist
[2020-09-22 15:47:22,365] WARNING - pip package requirement xgboost already exist
Out[13]:
Pclass Age Fare SibSp Parch pred
0 3 34.5 7.8292 0 0 0.341580
1 3 47.0 7.0000 1 0 0.341580
2 2 62.0 9.6875 0 0 0.371730
3 3 27.0 8.6625 0 0 0.341580
4 3 22.0 12.2875 1 1 0.341580
5 3 14.0 9.2250 0 0 0.341580
6 3 30.0 7.6292 0 0 0.341580
7 2 26.0 29.0000 1 1 0.472451
8 3 18.0 7.2292 0 0 0.341580
9 3 21.0 24.1500 2 0 0.341580
10 3 NaN 7.8958 0 0 0.341580
11 1 46.0 26.0000 0 0 0.413966
12 1 23.0 82.2667 1 0 0.469721
13 2 63.0 26.0000 1 0 0.413966
14 1 47.0 61.1750 1 0 0.445984
15 2 24.0 27.7208 1 0 0.437703
16 2 35.0 12.3500 0 0 0.371730
17 3 21.0 7.2250 0 0 0.341580
18 3 27.0 7.9250 1 0 0.341580
19 3 45.0 7.2250 0 0 0.341580
20 1 55.0 59.4000 1 0 0.445984
21 3 9.0 3.1708 0 1 0.341580
22 1 NaN 31.6833 0 0 0.413966
23 1 21.0 61.3792 0 1 0.469721
24 1 48.0 262.3750 1 3 0.445984
25 3 50.0 14.5000 1 0 0.341580
26 1 22.0 61.9792 0 1 0.469721
27 3 22.5 7.2250 0 0 0.341580
28 1 41.0 30.5000 0 0 0.437703
29 3 NaN 21.6792 2 0 0.341580
... ... ... ... ... ... ...
388 3 21.0 7.7500 0 0 0.341580
389 3 6.0 21.0750 3 1 0.329523
390 1 23.0 93.5000 0 0 0.469721
391 1 51.0 39.4000 0 1 0.448714
392 3 13.0 20.2500 0 2 0.341580
393 2 47.0 10.5000 0 0 0.371730
394 3 29.0 22.0250 3 1 0.341580
395 1 18.0 60.0000 1 0 0.469721
396 3 24.0 7.2500 0 0 0.341580
397 1 48.0 79.2000 1 1 0.445984
398 3 22.0 7.7750 0 0 0.341580
399 3 31.0 7.7333 0 0 0.341580
400 1 30.0 164.8667 0 0 0.469721
401 2 38.0 21.0000 1 0 0.437703
402 1 22.0 59.4000 0 1 0.469721
403 1 17.0 47.1000 0 0 0.437703
404 1 43.0 27.7208 1 0 0.413966
405 2 20.0 13.8625 0 0 0.437703
406 2 23.0 10.5000 1 0 0.371730
407 1 50.0 211.5000 1 1 0.445984
408 3 NaN 7.7208 0 0 0.341580
409 3 3.0 13.7750 1 1 0.471721
410 3 NaN 7.7500 0 0 0.341580
411 1 37.0 90.0000 1 0 0.469721
412 3 28.0 7.7750 0 0 0.341580
413 3 NaN 8.0500 0 0 0.341580
414 1 39.0 108.9000 0 0 0.469721
415 3 38.5 7.2500 0 0 0.341580
416 3 NaN 8.0500 0 0 0.341580
417 3 NaN 22.3583 1 1 0.341580

418 rows × 6 columns

Launch inference job from CLI

BentoML cli supports loading and running a packaged model from CLI. With the DataframeInput adapter, the CLI command supports reading input Dataframe data from CLI argument or local csv or json files:

In [14]:
!bentoml run TitanicSurvivalPredictionXgBoost:latest predict \
    --input '[{"Pclass": 1, "Age": 30, "Fare": 200, "SibSp": 1, "Parch": 0}]' 

[2020-09-22 15:47:27,495] INFO - Getting latest version TitanicSurvivalPredictionXgBoost:20200922154223_4ACDA4
[2020-09-22 15:47:27,533] WARNING - Using BentoML installed in `editable` model, the local BentoML repository including all code changes will be packaged together with saved bundle created, under the './bundled_pip_dependencies' directory of the saved bundle.
[2020-09-22 15:47:27,549] WARNING - Saved BentoService bundle version mismatch: loading BentoService bundle create with BentoML version 0.9.0.pre, but loading from BentoML version 0.9.0.pre+3.gcebf2015
[2020-09-22 15:47:28,744] INFO - Using default docker base image: `None` specified inBentoML config file or env var. User must make sure that the docker base image either has Python 3.7 or conda installed.
[15:47:28] WARNING: /Users/travis/build/dmlc/xgboost/src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[2020-09-22 15:47:31,407] INFO - {'service_name': 'TitanicSurvivalPredictionXgBoost', 'service_version': '20200922154223_4ACDA4', 'api': 'predict', 'task': {'data': {}, 'task_id': 'acc8e2a3-9ce7-4a18-aa34-164db3877cd0', 'batch': 1, 'cli_args': ('--input', '[{"Pclass": 1, "Age": 30, "Fare": 200, "SibSp": 1, "Parch": 0}]')}, 'result': {'data': '[0.469721257686615]', 'http_status': 200, 'http_headers': (('Content-Type', 'application/json'),)}, 'request_id': 'acc8e2a3-9ce7-4a18-aa34-164db3877cd0'}
[0.469721257686615]

Deployment Options

If you are at a small team with limited engineering or DevOps resources, try out automated deployment with BentoML CLI, currently supporting AWS Lambda, AWS SageMaker, and Azure Functions:

If the cloud platform you are working with is not on the list above, try out these step-by-step guide on manually deploying BentoML packaged model to cloud platforms:

Lastly, if you have a DevOps or ML Engineering team who's operating a Kubernetes or OpenShift cluster, use the following guides as references for implementating your deployment strategy:

In [ ]: