Kubernetes入门(四)——如何在Kubernetes中部署一个可对外服务的Tensorflow机器学习模型
机器学习模型常用Docker部署，而如何对Docker部署的模型进行管理呢？工业界的解决方案是使用Kubernetes来管理、编排容器。Kubernetes的理论知识不是本文讨论的重点，这里不再赘述，有关Kubernetes的优点读者可自行Google。笔者整理的Kubernetes入门系列的侧重点是如何实操，前三节介绍了Kubernets的安装、Dashboard的安装，以及如何在Kubernetes中部署一个无状态的应用，本节将讨论如何在Kubernetes中部署一个可对外服务的Tensorflow机器学习模型，作为Kubernetes入门系列的结尾。
希望Kubernetes入门系列能对K8S初学者提供一些参考，对文中描述有不同观点，或者对工业级部署与应用机器学习算法模型有什么建议，欢迎大家在评论区讨论与交流~~~
1. Docker中运行TensorFolw Serving
- 运行half_plus_two模型 [1]
# Download the TensorFlow Serving Docker image and repo
docker pull tensorflow/serving
mkdir /data0/modules
cd /data0/modules
git clone https://github.com/tensorflow/serving
# Location of demo models
TESTDATA="/data0/modules/serving/tensorflow_serving/servables/tensorflow/testdata/"
# Start TensorFlow Serving container and open the REST API port
docker run -dit --rm -p 8501:8501 \
-v /data0/modules/serving/tensorflow_serving/servables/tensorflow/testdata/saved_model_half_plus_two_cpu:/models/half_plus_two \
-e MODEL_NAME=half_plus_two tensorflow/serving
# Query the model using the predict API
curl -d '{"instances": [1.0, 2.0, 5.0]}' \
-X POST http://localhost:8501/v1/models/half_plus_two:predict
# Returns => { "predictions": [2.5, 3.0, 4.5] }
2. 构建TensorFolw模型的Docker镜像
- 后台运行serving容器
docker run -d --rm --name serving_base tensorflow/serving
- 拷贝模型数据到容器中的model目录
docker cp /data0/modules/serving/tensorflow_serving/servables/tensorflow/testdata/saved_model_half_plus_two_cpu serving_base:/models/half_plus_two
- 生成关于模型的镜像
docker commit --change "ENV MODEL_NAME half_plus_two" serving_base ljh/half_plus_two
- 停止serving容器
docker kill serving_base
docker rm serving_base
- 启动服务
docker run -dit --rm -p 8501:8501 \
-e MODEL_NAME=half_plus_two ljh/half_plus_two
- 查询模型
curl -d '{"instances": [1.0, 2.0, 5.0]}' -X POST http://localhost:8501/v1/models/half_plus_two:predict
# Returns => { "predictions": [2.5, 3.0, 4.5] }
3. Kubernetes部署TensorFolw模型
创建关于模型的Deployment
- yaml文件
cat deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: halfplustwo-deployment
spec:
selector:
matchLabels:
app: halfplustwo
replicas: 1
template:
metadata:
labels:
app: halfplustwo
spec:
containers:
- name: halfplustwo
image: ljh/half_plus_two:latest
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8501
name: restapi
- containerPort: 8500
name: grpc
- 创建一个Deployment:
kubectl apply -f deployment.yaml
- 展示Deployment相关信息:
kubectl get deployment -o wide
kubectl describe deployment halfplustwo-deployment
- 列出deployment创建的pods:
kubectl get pods -l app=halfplustwo
- 展示某一个pod信息
kubectl describe pod <pod-name>
使用service暴露你的应用
- yaml文件
cat service.yaml
apiVersion: v1
kind: Service
metadata:
labels:
run: halfplustwo-service
name: halfplustwo-service
spec:
ports:
- port: 8501
targetPort: 8501
name: restapi
- port: 8500
targetPort: 8500
name: grpc
selector:
app: halfplustwo
type: LoadBalancer
- 启动service
kubectl create -f service.yaml
or
kubectl apply -f service.yaml
- 查看service
kubectl get service
#output:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
halfplustwo-service LoadBalancer 10.96.181.116 <pending> 8501:30771/TCP,8500:31542/TCP 4s
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 8d
nginx NodePort 10.96.153.10 <none> 80:30088/TCP 29h
测试
curl -d '{"instances": [1.0, 2.0, 5.0]}' -X POST http://localhost:8501/v1/models/half_plus_two:predict
{"predictions": [2.5, 3.0, 4.5]}
删除deployment和service
kubectl delete -f deployment.yaml
kubectl delete -f service.yaml
4. 参考资料
[1] https://www.tensorflow.org/tfx/serving/docker TensorFlow Serving 与 Docker
[2] https://www.tensorflow.org/tfx/serving/serving_kubernetes?hl=zh_cn 将TensorFlow Serving与 Kubernetes结合使用
[3] https://towardsdatascience.com/scaling-machine-learning-models-using-tensorflow-serving-kubernetes-ed00d448c917 Scaling Machine Learning models using Tensorflow Serving & Kubernetes
[4] http://www.tuwee.cn/2019/03/03/Kubernetes+Tenserflow-serving%E6%90%AD%E5%BB%BA%E5%8F%AF%E5%AF%B9%E5%A4%96%E6%9C%8D%E5%8A%A1%E7%9A%84%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0%E5%BA%94%E7%94%A8/ Kubernetes+Tenserflow-serving搭建可对外服务的机器学习应用
