BentoML Demo: Titanic Survival Prediction with XGBoost

BentoML is an open-source framework for high-performance ML model serving.

This notebook demonstrates use BentoML to serve a model trained with the XGBoost framework, specifically using the Titanic Survival dataset.

Let's get started! Impression

In [1]:
%reload_ext autoreload
%autoreload 2
%matplotlib inline

import warnings
warnings.filterwarnings("ignore")
In [2]:
!pip install -q --upgrade xgboost==0.90 numpy==1.18.5 pandas==1.0.4 bentoml
In [3]:
import pandas as pd
import numpy as np
import xgboost as xgb
import bentoml

Prepare Dataset

download dataset from https://www.kaggle.com/c/titanic/data

In [4]:
!mkdir data
!curl https://raw.githubusercontent.com/agconti/kaggle-titanic/master/data/train.csv -o ./data/train.csv
!curl https://raw.githubusercontent.com/agconti/kaggle-titanic/master/data/test.csv -o ./data/test.csv
mkdir: data: File exists
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 60302  100 60302    0     0   180k      0 --:--:-- --:--:-- --:--:--  180k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 28210  100 28210    0     0  99330      0 --:--:-- --:--:-- --:--:-- 99330
In [5]:
train = pd.read_csv("./data/train.csv")
test  = pd.read_csv("./data/test.csv")
X_y_train = xgb.DMatrix(data=train[['Pclass', 'Age', 'Fare', 'SibSp', 'Parch']], label= train['Survived'])
X_test    = xgb.DMatrix(data=test[['Pclass', 'Age', 'Fare', 'SibSp', 'Parch']])
In [6]:
train[['Pclass', 'Age', 'Fare', 'SibSp', 'Parch', 'Survived']].head()
Out[6]:
Pclass Age Fare SibSp Parch Survived
0 3 22.0 7.2500 1 0 0
1 1 38.0 71.2833 1 0 1
2 3 26.0 7.9250 0 0 1
3 1 35.0 53.1000 1 0 1
4 3 35.0 8.0500 0 0 0

Model Training

In [7]:
params = {
          'base_score': np.mean(train['Survived']),
          'eta':  0.1,
          'max_depth': 3,
          'gamma' :3,
          'objective'   :'reg:linear',
          'eval_metric' :'mae'
         }
model = xgb.train(params=params, 
                  dtrain=X_y_train, 
                  num_boost_round=3)
[17:27:18] WARNING: src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.
In [8]:
y_test =  model.predict(X_test)
test['pred'] = y_test
test[['Pclass', 'Age', 'Fare', 'SibSp', 'Parch','pred']].iloc[10:].head(2)
Out[8]:
Pclass Age Fare SibSp Parch pred
10 3 NaN 7.8958 0 0 0.341580
11 1 46.0 26.0000 0 0 0.413966

Create BentoService for model serving

In [9]:
%%writefile xgboost_titanic_bento_service.py

import xgboost as xgb

import bentoml
from bentoml.artifact import XgboostModelArtifact
from bentoml.adapters import DataframeInput

@bentoml.env(auto_pip_dependencies=True)
@bentoml.artifacts([XgboostModelArtifact('model')])
class TitanicSurvivalPredictionXgBoost(bentoml.BentoService):
    
    @bentoml.api(input=DataframeInput())
    def predict(self, df):
        data = xgb.DMatrix(data=df[['Pclass', 'Age', 'Fare', 'SibSp', 'Parch']])
        return self.artifacts.model.predict(data)
Overwriting xgboost_titanic_bento_service.py

Create BentoService saved bundle

In [10]:
# 1) import the custom BentoService defined above
from xgboost_titanic_bento_service import TitanicSurvivalPredictionXgBoost

# 2) `pack` it with required artifacts
bento_service = TitanicSurvivalPredictionXgBoost()
bento_service.pack('model', model)

# 3) save your BentoSerivce
saved_path = bento_service.save()
[2020-06-16 17:27:43,973] INFO - BentoService bundle 'TitanicSurvivalPredictionXgBoost:20200616172725_4CD6B0' saved to: /Users/chaoyu/bentoml/repository/TitanicSurvivalPredictionXgBoost/20200616172725_4CD6B0

Load BentoService saved bundle

In [11]:
import bentoml

loaded_svc = bentoml.load(saved_path)

result = loaded_svc.predict(test)
test['pred'] = result
test[['Pclass', 'Age', 'Fare', 'SibSp', 'Parch','pred']]
[2020-06-16 17:27:44,282] WARNING - Module `xgboost_titanic_bento_service` already loaded, using existing imported module.
[17:27:44] WARNING: src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.
Out[11]:
Pclass Age Fare SibSp Parch pred
0 3 34.5 7.8292 0 0 0.341580
1 3 47.0 7.0000 1 0 0.341580
2 2 62.0 9.6875 0 0 0.371730
3 3 27.0 8.6625 0 0 0.341580
4 3 22.0 12.2875 1 1 0.341580
... ... ... ... ... ... ...
413 3 NaN 8.0500 0 0 0.341580
414 1 39.0 108.9000 0 0 0.469721
415 3 38.5 7.2500 0 0 0.341580
416 3 NaN 8.0500 0 0 0.341580
417 3 NaN 22.3583 1 1 0.341580

418 rows × 6 columns

Working with BentoML CLI

bentoml get <BentoServiceName> is great for list all versions of the BentoService

In [12]:
!bentoml get TitanicSurvivalPredictionXgBoost
BENTO_SERVICE                                           AGE                           APIS                                   ARTIFACTS
TitanicSurvivalPredictionXgBoost:20200616172725_4CD6B0  2.68 seconds                  predict<DataframeInput:DefaultOutput>  model<XgboostModelArtifact>
TitanicSurvivalPredictionXgBoost:20200616170508_6AF596  22 minutes and 19.01 seconds  predict<DataframeInput:DefaultOutput>  model<XgboostModelArtifact>
TitanicSurvivalPredictionXgBoost:20200616163101_75E9BA  56 minutes and 26.2 seconds   predict<DataframeInput:DefaultOutput>  model<XgboostModelArtifact>

bentoml get <BentoService name>:<BentoService version> to access detailed information

In [13]:
!bentoml get TitanicSurvivalPredictionXgBoost:latest
[2020-06-16 17:27:48,704] INFO - Getting latest version TitanicSurvivalPredictionXgBoost:20200616172725_4CD6B0
{
  "name": "TitanicSurvivalPredictionXgBoost",
  "version": "20200616172725_4CD6B0",
  "uri": {
    "type": "LOCAL",
    "uri": "/Users/chaoyu/bentoml/repository/TitanicSurvivalPredictionXgBoost/20200616172725_4CD6B0"
  },
  "bentoServiceMetadata": {
    "name": "TitanicSurvivalPredictionXgBoost",
    "version": "20200616172725_4CD6B0",
    "createdAt": "2020-06-17T00:27:43.917769Z",
    "env": {
      "condaEnv": "name: bentoml-TitanicSurvivalPredictionXgBoost\nchannels:\n- defaults\ndependencies:\n- python=3.7.5\n- pip\n",
      "pipDependencies": "xgboost==0.90\npandas\nbentoml==0.8.1",
      "pythonVersion": "3.7.5",
      "dockerBaseImage": "bentoml/model-server:0.8.1"
    },
    "artifacts": [
      {
        "name": "model",
        "artifactType": "XgboostModelArtifact"
      }
    ],
    "apis": [
      {
        "name": "predict",
        "inputType": "DataframeInput",
        "docs": "BentoService API",
        "inputConfig": {
          "orient": "records",
          "typ": "frame",
          "is_batch_input": true,
          "input_dtypes": null
        },
        "outputConfig": {
          "cors": "*"
        },
        "outputType": "DefaultOutput"
      }
    ]
  }
}

Run predicition task from CLI:

In [14]:
!bentoml run TitanicSurvivalPredictionXgBoost:latest predict \
    --input '[{"Pclass": 1, "Age": 30, "Fare": 200, "SibSp": 1, "Parch": 0}]' 
[2020-06-16 17:27:50,818] INFO - Getting latest version TitanicSurvivalPredictionXgBoost:20200616172725_4CD6B0
[17:27:51] WARNING: src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.
[0.46972126]

Model Serving via REST API

Start local API model server with bentoml serve command:

In [15]:
!bentoml serve TitanicSurvivalPredictionXgBoost:latest
[2020-06-16 17:27:58,816] INFO - Getting latest version TitanicSurvivalPredictionXgBoost:20200616172725_4CD6B0
[17:27:59] WARNING: src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.
 * Serving Flask app "TitanicSurvivalPredictionXgBoost" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
127.0.0.1 - - [16/Jun/2020 17:28:01] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [16/Jun/2020 17:28:01] "GET /docs.json HTTP/1.1" 200 -
127.0.0.1 - - [16/Jun/2020 17:28:14] "POST /predict HTTP/1.1" 200 -
127.0.0.1 - - [16/Jun/2020 17:28:17] "POST /predict HTTP/1.1" 200 -
127.0.0.1 - - [16/Jun/2020 17:28:17] "POST /predict HTTP/1.1" 200 -
127.0.0.1 - - [16/Jun/2020 17:28:17] "POST /predict HTTP/1.1" 200 -
127.0.0.1 - - [16/Jun/2020 17:28:17] "POST /predict HTTP/1.1" 200 -
^C

Copy following curl command to make a curl request to REST API server from another terminal window:

curl -i \
--header "Content-Type: application/json" \
--request POST \
--data '[{"Pclass": 1, "Age": 30, "Fare": 200, "SibSp": 1, "Parch": 0}]' \
localhost:5000/predict

Containerize REST API server with Docker

The BentoService saved bundle direcotry is structured as a docker build context, which can be used directly to build a docker image for the model server:

In [16]:
!cd {saved_path} && docker build --quiet -t titanic-survival-predict-server .
sha256:986cbfc72fbca0645a54267df982f4970df5fb408d388fd9f2c7502084d7a686

Next, you can docker push the image to your choice of registry for deployment, or run it locally for development and testing:

In [17]:
!docker run -p 5000:5000 titanic-survival-predict-server
[2020-06-17 00:29:53,084] INFO - get_gunicorn_num_of_workers: 3, calculated by cpu count
[2020-06-17 00:29:53 +0000] [1] [INFO] Starting gunicorn 20.0.4
[2020-06-17 00:29:53 +0000] [1] [INFO] Listening at: http://0.0.0.0:5000 (1)
[2020-06-17 00:29:53 +0000] [1] [INFO] Using worker: sync
[2020-06-17 00:29:53 +0000] [12] [INFO] Booting worker with pid: 12
[2020-06-17 00:29:53 +0000] [13] [INFO] Booting worker with pid: 13
[2020-06-17 00:29:53 +0000] [14] [INFO] Booting worker with pid: 14
^C
[2020-06-17 00:34:02 +0000] [1] [INFO] Handling signal: int
[2020-06-17 00:34:02 +0000] [14] [INFO] Worker exiting (pid: 14)
[2020-06-17 00:34:02 +0000] [13] [INFO] Worker exiting (pid: 13)
[2020-06-17 00:34:02 +0000] [12] [INFO] Worker exiting (pid: 12)
[00:29:53] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.
[00:29:53] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.
[00:29:53] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.

Deploy BentoService as REST API server to the cloud

BentoML support deployment to multiply cloud provider services, such as AWS Lambda, AWS Sagemaker, Google Cloudrun and etc. You can find the full list and guide on the documentation site at https://docs.bentoml.org/en/latest/deployment/index.html

Here is an example of deploying this saved BentoService to AWS Lambda.

Before started:

In [39]:
!bentoml lambda deploy xgb-titanic-survival-predict \
    -b TitanicSurvivalPredictionXgBoost:{bento_service.version}
Deploying "TitanicSurvivalPredictionXgBoost:20200616172725_4CD6B0" to AWS Lambda /[2020-06-16 18:04:35,838] INFO - Building lambda project
\[2020-06-16 18:09:26,915] INFO - Packaging AWS Lambda project at /private/var/folders/7p/y_934t3s4yg8fx595vr28gym0000gn/T/bentoml-temp-8ddc4h4b ...
/[2020-06-16 18:12:08,657] INFO - Deploying lambda project
\[2020-06-16 18:13:01,474] INFO - ApplyDeployment (xgb-titanic-survival-predict, namespace dev) succeeded
Successfully created AWS Lambda deployment xgb-titanic-survival-predict
{
  "namespace": "dev",
  "name": "xgb-titanic-survival-predict",
  "spec": {
    "bentoName": "TitanicSurvivalPredictionXgBoost",
    "bentoVersion": "20200616172725_4CD6B0",
    "operator": "AWS_LAMBDA",
    "awsLambdaOperatorConfig": {
      "region": "us-west-2",
      "memorySize": 1024,
      "timeout": 3
    }
  },
  "state": {
    "state": "RUNNING",
    "infoJson": {
      "endpoints": [
        "https://o2quazkrli.execute-api.us-west-2.amazonaws.com/Prod/predict"
      ],
      "s3_bucket": "btml-dev-xgb-titanic-survival-predict-22ec76"
    },
    "timestamp": "2020-06-17T01:13:01.701051Z"
  },
  "createdAt": "2020-06-17T01:04:30.344490Z",
  "lastUpdatedAt": "2020-06-17T01:04:30.344520Z"
}
In [40]:
!bentoml lambda get xgb-titanic-survival-predict
{
  "namespace": "dev",
  "name": "xgb-titanic-survival-predict",
  "spec": {
    "bentoName": "TitanicSurvivalPredictionXgBoost",
    "bentoVersion": "20200616172725_4CD6B0",
    "operator": "AWS_LAMBDA",
    "awsLambdaOperatorConfig": {
      "region": "us-west-2",
      "memorySize": 1024,
      "timeout": 3
    }
  },
  "state": {
    "state": "RUNNING",
    "infoJson": {
      "endpoints": [
        "https://o2quazkrli.execute-api.us-west-2.amazonaws.com/Prod/predict"
      ],
      "s3_bucket": "btml-dev-xgb-titanic-survival-predict-22ec76"
    },
    "timestamp": "2020-06-17T01:13:04.861472Z"
  },
  "createdAt": "2020-06-17T01:04:30.344490Z",
  "lastUpdatedAt": "2020-06-17T01:04:30.344520Z"
}

To send request to your AWS Lambda deployment, grab the endpoint URL from the json output above:

In [ ]:
!curl -i \
    --header "Content-Type: application/json" \
    --request POST \
    --data '[{"Pclass": 1, "Age": 30, "Fare": 200, "SibSp": 1, "Parch": 0}]' \
    $(bentoml lambda get xgb-titanic-survival-predict | jq -r ".state.infoJson.endpoints[0]")

Use bentoml lambda delete to remove AWS Lambda deployment:

In [38]:
!bentoml lambda delete xgb-titanic-survival-predict
Successfully deleted AWS Lambda deployment "xgb-titanic-survival-predict"
In [ ]: