kubernetes ingress金丝雀发布/灰度发布
1.金丝雀发布介绍
灰度发布(又名金丝雀发布)是指在黑与白之间,能够平滑过渡的一种发布方式。在其上可以进行A/B testing,即让一部分用户继续用产品特性A,一部分用户开始用产品特性B
蓝绿发布是部署一套新版本的环境,待充分测试验证以后再发布,这样比较消耗资源,所以可以先更新一部分pod或者一定比例的pod,新版本的pod运行一段时间后,如果没有报错,那么就可以逐步扩大新版本的pod的数量,并逐步完成更新。
所以可以先更新一部分pod或者一定比例的pod,新版本的pod运行一段时间后,如果没有报错,那么就可以逐步扩大新版本的pod的数量,并逐步完成更新
Ingress-Nginx 是一个K8S ingress工具,支持配置 Ingress Annotations 来实现不同场景下的灰度发布和测试。 Nginx Annotations 支持以下 4 种 Canary 规则:
nginx.ingress.kubernetes.io/canary-by-header:基于 Request Header 的流量切分,适用于灰度发布以及 A/B 测试。当 Request Header 设置为 always时,请求将会被一直发送到 Canary 版本;当 Request Header 设置为 never时,请求不会被发送到 Canary 入口;对于任何其他 Header 值,将忽略 Header,并通过优先级将请求与其他金丝雀规则进行优先级的比较。
nginx.ingress.kubernetes.io/canary-by-header-value:要匹配的 Request Header 的值,用于通知 Ingress 将请求路由到 Canary Ingress 中指定的服务。当 Request Header 设置为此值时,它将被路由到 Canary 入口。该规则允许用户自定义 Request Header 的值,必须与上一个 annotation (即:canary-by-header)一起使用。
nginx.ingress.kubernetes.io/canary-weight:基于服务权重的流量切分,适用于蓝绿部署,权重范围 0 - 100 按百分比将请求路由到 Canary Ingress 中指定的服务。权重为 0 意味着该金丝雀规则不会向 Canary 入口的服务发送任何请求。权重为 100 意味着所有请求都将被发送到 Canary 入口。
nginx.ingress.kubernetes.io/canary-by-cookie:基于 Cookie 的流量切分,适用于灰度发布与 A/B 测试。用于通知 Ingress 将请求路由到 Canary Ingress 中指定的服务的cookie。当 cookie 值设置为 always时,它将被路由到 Canary 入口;当 cookie 值设置为 never时,请求不会被发送到 Canary 入口;对于任何其他值,将忽略 cookie 并将请求与其他金丝雀规则进行优先级的比较。
注意:金丝雀规则按优先顺序进行如下排序:
canary-by-header - > canary-by-cookie - > canary-weight
我们可以把以上的四个 annotation 规则可以总体划分为以下两类:
- 基于权重的 Canary 规则:
- 基于用户请求的 Canary 规则:
注意: Ingress-Nginx 实在0.21.0 版本 中,引入的Canary 功能,因此要确保ingress版本OK
本文演示通过nginx.ingress.kubernetes.io/canary-weight进行金丝雀测试
2.部署green的service
green_service.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-green
spec:
replicas: 4
selector:
matchLabels:
app: nginx-green
template:
metadata:
labels:
app: nginx-green
spec:
containers:
- name: nginx-green
image: nginx
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
volumeMounts:
- name: www
mountPath: /usr/share/nginx/html
volumes:
- name: www
configMap:
name: nginx-green
---
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-green
data:
index.html: |
green green green --> If you see this message --> The route to green,bye ~~~;
---
apiVersion: v1
kind: Service
metadata:
name: nginx-green
labels:
app: nginx-green
spec:
ports:
- port: 80
targetPort: 80
protocol: TCP
name: http
selector:
app: nginx-green
部署:
kubectl apply -f green_service.yaml
查看pods:
qiteck@server:~/program/k8s/blue_green_split$ sudo kubectl get pod
nginx-green-78968cd575-bngtx 1/1 Running 0 79m
nginx-green-78968cd575-cv986 1/1 Running 0 79m
nginx-green-78968cd575-pkpcr 1/1 Running 0 79m
nginx-green-78968cd575-xz4dj 1/1 Running 0 79m
查看service:
qiteck@server:~/program/k8s/blue_green_split$ sudo kubectl get service
nginx-green ClusterIP 10.96.255.252 <none> 80/TCP 79m
3. 部署blue的service
blue_service.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-blue
spec:
replicas: 4
selector:
matchLabels:
app: nginx-blue
template:
metadata:
labels:
app: nginx-blue
spec:
containers:
- name: nginx-blue
image: nginx
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
volumeMounts:
- name: www
mountPath: /usr/share/nginx/html
volumes:
- name: www
configMap:
name: nginx-blue
---
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-blue
data:
index.html: |
blue blue blue --> If you see this message --> The route to blue,bye ~~~;
---
apiVersion: v1
kind: Service
metadata:
name: nginx-blue
labels:
app: nginx-blue
spec:
ports:
- port: 80
targetPort: 80
protocol: TCP
name: http
selector:
app: nginx-blue
部署:
kubectl apply -f blue_service.yaml
查看pods:
qiteck@server:~/program/k8s/blue_green_split$ sudo kubectl get pods |grep nginx
nginx-blue-766b9f6df4-4ldzx 1/1 Running 0 38m
nginx-blue-766b9f6df4-bmd85 1/1 Running 0 38m
nginx-blue-766b9f6df4-h5bdd 1/1 Running 0 38m
nginx-blue-766b9f6df4-qsn5w 1/1 Running 0 38m
nginx-green-78968cd575-bngtx 1/1 Running 0 97m
nginx-green-78968cd575-cv986 1/1 Running 0 97m
nginx-green-78968cd575-pkpcr 1/1 Running 0 97m
nginx-green-78968cd575-xz4dj 1/1 Running 0 97m
查看service:
qiteck@server:~/program/k8s/blue_green_split$ sudo kubectl get service |grep nginx
nginx-blue ClusterIP 10.96.95.252 <none> 80/TCP 39m
nginx-green ClusterIP 10.96.255.252 <none> 80/TCP 98m
目前可以看到blue和green两套服务都是存在的
4.部署green的ingress
green_ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-green
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: nginx.test.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx-green
port:
number: 80
域名nginx.test.com映射到nginx_green服务:
部署:
sudo kubectl apply -f ingress.yaml
查看ingress:
qiteck@server:~/program/k8s/blue_green_split$ sudo kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
nginx <none> nginx.test.com 10.96.180.91 80 49s
5.部署blue的ingress
blue_ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-blue
annotations:
kubernetes.io/ingress.class: nginx #指定ingress服务,无意义,可不写
nginx.ingress.kubernetes.io/canary: "true" #打开金丝雀开关
nginx.ingress.kubernetes.io/canary-weight: "50" #权重,按照50%的流量区分
spec:
rules:
- host: nginx.test.com #和上面一样的域名
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx-blue #请求转到什么后端
port:
number: 80
这里设置了金丝雀开发和weight:
nginx.ingress.kubernetes.io/canary: "true" #打开金丝雀开关
nginx.ingress.kubernetes.io/canary-weight: "50" #权重,按照50%的流量区分
域名nginx.test.com映射到nginx_blue服务:
部署:
sudo kubectl apply -f ingress.yaml
查看ingress:
qiteck@server:~/program/k8s/blue_green_split$ sudo kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
nginx-blue <none> nginx.test.com 10.96.180.91 80 23m
nginx-green <none> nginx.test.com 10.96.180.91 80 37m
可以看到nginx.test.com同时影射到nginx-blue和nginx-green2服务了。
如果不设置canary的话,会提示域名和path已经存在
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-blue
annotations:
kubernetes.io/ingress.class: nginx #指定ingress服务,无意义,可不写
spec:
rules:
- host: nginx.yemaodiyi.com #和上面一样的域名
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx-blue #请求转到什么后端
port:
number: 80
一个path只能定位到一个service:
6.测试一下金丝雀权重
qiteck@server:~/program/k8s/blue_green_split$ for i in `seq 20`;do curl nginx.test.com;done
green green green --> If you see this message --> The route to green,bye ~~~;
blue blue blue --> If you see this message --> The route to blue,bye ~~~;
blue blue blue --> If you see this message --> The route to blue,bye ~~~;
green green green --> If you see this message --> The route to green,bye ~~~;
green green green --> If you see this message --> The route to green,bye ~~~;
green green green --> If you see this message --> The route to green,bye ~~~;
blue blue blue --> If you see this message --> The route to blue,bye ~~~;
blue blue blue --> If you see this message --> The route to blue,bye ~~~;
blue blue blue --> If you see this message --> The route to blue,bye ~~~;
green green green --> If you see this message --> The route to green,bye ~~~;
green green green --> If you see this message --> The route to green,bye ~~~;
blue blue blue --> If you see this message --> The route to blue,bye ~~~;
green green green --> If you see this message --> The route to green,bye ~~~;
green green green --> If you see this message --> The route to green,bye ~~~;
green green green --> If you see this message --> The route to green,bye ~~~;
blue blue blue --> If you see this message --> The route to blue,bye ~~~;
green green green --> If you see this message --> The route to green,bye ~~~;
green green green --> If you see this message --> The route to green,bye ~~~;
green green green --> If you see this message --> The route to green,bye ~~~;
blue blue blue --> If you see this message --> The route to blue,bye ~~~;
可以看到blue和green同时在提供服务了。
注意这里权重不是一个精确的百分比,使用过程当中,只是会看到一个近似分布。