【kserve】kf-serving预测模型使用教程

lightgbm

启动脚本：

apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
  name: "lightgbm-iris"
spec:
  predictor:
    lightgbm:
      storageUri: "gs://kfserving-examples/models/lightgbm/iris"

预测:

http://kf00:30000/v1/models/lightgbm-iris:predict

{
  "inputs": [{
    "sepal_length_(cm)": [5.1],
    "sepal_width_(cm)": [3.5],
    "petal_length_(cm)": [1.4],
    "petal_width_(cm)": [0.2]
  }]
}

 

tensorflow

启动脚本:

apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
  labels:
    predict-phantom.wenyangchou.com/modelName: "lgb-test"
    predict-phantom.wenyangchou.com/modelVersion: "v1.2"
  name: "t"
spec:
  predictor:
    maxReplicas: 1
    minReplicas: 1
    canaryTrafficPercent: 100
    tensorflow:
      resources:
        limits:
          memory: "4Gi"
          cpu: "8"
        requests:
          memory: "4Gi"
          cpu: "8"
      runtimeVersion: 2.3.0
      storageUri: "http://www.wenyangchou.com/nfs/t-model.zip"

模型信息接口:

跟tensorflow-serving一致，地址为http://kf01:30000/v1/models/t/metadata

host:t.default.wenyangchou.com

预测接口:

http://kf01:30000/v1/models/t:predict

参数:

{
    "inputs":{
        "x_k1":[[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 405, 405, 405, 430, 430, 430, 430, 430, 430, 430, 430, 430, 430, 462, 462, 462, 462, 462, 462, 462, 462, 462, 462, 679, 679, 679, 679, 679]],
        "x_k2":[[75, 76, 74, 75, 0, 0, 1, 0, 0, 171, 188, 261, 2, 405, 405, 33, 430, 430, 679, 679, 679]],
        "x_k3":[[0, 0, 405, 430, 462, 679, 679, 679]],
        "x_v1":[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]],
        "x_v2":[[100.0999984741211, 4.199999809265137, 0, 0]],
        "x_v3":[[0.0, 0.0, 0.0, 0.0, -1.0]],
        "is_training":false
    }
}

模型格式:

- 1

　　- saved_model.pb

　　- variables

pytorch

启动脚本：

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: "torchserve"
spec:
  predictor:
    pytorch:
      storageUri: gs://kfserving-examples/models/torchserve/image_classifier

模型预测/解释：

http://kf00:30000/v1/models/mnist:predict

http://kf00:30000/v1/models/mnist:explain

参数:

{
  "instances": [
    {
      "data": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAAAw0lEQVR4nGNgGFggVVj4/y8Q2GOR83n+58/fP0DwcSqmpNN7oOTJw6f+/H2pjUU2JCSEk0EWqN0cl828e/FIxvz9/9cCh1zS5z9/G9mwyzl/+PNnKQ45nyNAr9ThMHQ/UG4tDofuB4bQIhz6fIBenMWJQ+7Vn7+zeLCbKXv6z59NOPQVgsIcW4QA9YFi6wNQLrKwsBebW/68DJ388Nun5XFocrqvIFH59+XhBAxThTfeB0r+vP/QHbuDCgr2JmOXoSsAAKK7bU3vISS4AAAAAElFTkSuQmCC",
      "target": 0
    }
  ]
}

参考文档https://github.com/kserve/kserve/tree/master/docs/samples/v1beta1/torchserve

模型格式:

├── config
│   ├── config.properties
├── model-store
│   ├── densenet_161.mar
│   ├── mnist.mar

需要参考torchserve的模型导出，较复杂【TODO】

PMML

启动脚本

apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
  name: "pmml-demo"
spec:
  predictor:
    pmml:
      storageUri: http://www.wenyangchou.com/nfs/model.pmml

模型预测:

http://kf00:30000/v1/models/pmml-demo:predict

Host:pmml-demo.default.wenyangchou.com

请求参数:

{
  "instances":[[5.1, 3.5, 1.4, 0.2]] 
}

模型名必须是model.bst

 

返回参数:

{
    "inputs": [
        []
    ],

　　"cat_feature":[
　　　　"sp_fcat_mins_pt50_7d"
　　]

}

sklearn模型

当前sklearn支持两个版本

• 版本1 支持.pkl, .joblib, .pickle 三种格式，调用的predict方法。版本1的模型名必须是 model

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: test-sklearn
spec:
  predictor:
    sklearn:
      name: kserve-container
      protocolVersion: v1
      resources:
        limits:
          cpu: "2"
          memory: 2Gi
        requests:
          cpu: "2"
          memory: 2Gi
      runtimeVersion: v0.7.0
      storageUri: http://wenyangchou/nfs/model.joblib

　　

预测参数如下，key为instances，value为实际pkl的predict调用的　　

{
  "instances": [
    [
      0,
      6500,
      20
    ]
  ]
}

　　

• 版本2 只支持.joblib，底层使用的 SeldonIO 提供的镜像

apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
  name: "pkl-test"
spec:
  predictor:
    sklearn:
      protocolVersion: v2
      storageUri: "http://wenyangchou/model.pkl"

　　调用参数;

{
  "inputs": [
    {
      "name": "input-0",
      "shape": [2, 4],
      "datatype": "FP32",
      "data": [
        [6.8, 2.8, 4.8, 1.4],
        [6.0, 3.4, 4.5, 1.6]
      ]
    }
  ]
}

　　

 

自定义模型

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: embedding-test
spec:
  predictor:
    containers:
      - name: orochi
        image: kf05:5000/orochi/embedding:v1

需要注意的是，这个容器启动必须是一个rest服务

 

修改域名

https://knative.dev/docs/serving/using-a-custom-domain/

官网地址

https://kserve.github.io/website/modelserving/v1beta1/tensorflow/