k8s- ingress nginx -灰度发布 ，第二篇 实战

实验的环境：

1、已经搭建好k8s环境的3台 centos 7.9机器。1个master,2个worker节点。 k8s版本为1.20.4

2、在k8s里安装好  ingress  nginx,安装教程请看

《k8s- nginx ingress 高可用部署 1》  

《k8s- nginx ingress 高可用部署 2》 

《k8s- nginx ingress 高可用部署 3》 

 

实战

一、创建一个常规的应用

1、将镜像文件 openresty.tar.gz传入 2个workder节点，docker load 安装镜像；这个镜像相当于nginx的一个高级版本。

2、创建1个常规的应用，包括 Deployment,Service,Ingress资源。

v1.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
      version: v1
  template:
    metadata:
      labels:
        app: nginx
        version: v1
    spec:
      containers:
      - name: nginx
        image: "openresty/openresty:centos"
        imagePullPolicy: IfNotPresent
        ports:
        - name: http
          protocol: TCP
          containerPort: 80
        volumeMounts:
        - mountPath: /usr/local/openresty/nginx/conf/nginx.conf
          name: config
          subPath: nginx.conf
      volumes:
      - name: config
        configMap:
          name: nginx-v1
---
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app: nginx
    version: v1
  name: nginx-v1
data:
  nginx.conf: |-
    worker_processes  1;
    events {
        accept_mutex on;
        multi_accept on;
        use epoll;
        worker_connections  1024;
    }
    http {
        ignore_invalid_headers off;
        server {
            listen 80;
            location / {
                access_by_lua '
                    local header_str = ngx.say("nginx-v1")
                ';
            }
        }
    }
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-v1
spec:
  type: ClusterIP
  ports:
  - port: 80
    protocol: TCP
    name: http
  selector:
    app: nginx
    version: v1
View Code

v1-ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - host: canary.example.com
    http:
      paths:
      - backend:
          serviceName: nginx-v1
          servicePort: 80
        path: /
View Code

从yaml文件中，我们可以看到，创建了一个应用,标签 app为nginx,version为v1 .

 只返回一句话 "nginx-v1" 。访问的域名为 ：canary.example.com

kubectl apply -f 执行这2个yaml文件，等待1分钟左右。

 

3、访问验证一下:
$ curl -H "Host: canary.example.com" http://EXTERNAL-IP # EXTERNAL-IP 替换为Nginx Ingress 自身对外暴露的 IP

我的实验环境为：

curl -H "Host: canary.example.com" http://192.168.24.21

返回 nginx-v1

 

4、创建一个灰度版本（金丝雀版本）的资源 ，包 Deploy,Service

v2.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
      version: v2
  template:
    metadata:
      labels:
        app: nginx
        version: v2
    spec:
      containers:
      - name: nginx
        image: "openresty/openresty:centos"
        imagePullPolicy: IfNotPresent
        ports:
        - name: http
          protocol: TCP
          containerPort: 80
        volumeMounts:
        - mountPath: /usr/local/openresty/nginx/conf/nginx.conf
          name: config
          subPath: nginx.conf
      volumes:
      - name: config
        configMap:
          name: nginx-v2
---
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app: nginx
    version: v2
  name: nginx-v2
data:
  nginx.conf: |-
    worker_processes  1;
    events {
        accept_mutex on;
        multi_accept on;
        use epoll;
        worker_connections  1024;
    }
    http {
        ignore_invalid_headers off;
        server {
            listen 80;
            location / {
                access_by_lua '
                    local header_str = ngx.say("nginx-v2")
                ';
            }
        }
    }
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-v2
spec:
  type: ClusterIP
  ports:
  - port: 80
    protocol: TCP
    name: http
  selector:
    app: nginx
    version: v2
View Code

从yaml文件中，我们可以看到，创建了一个应用,标签 app为nginx,version为v2.

 只返回一句话 "nginx-v2" 。

执行v2.yaml

 

 

二、基于 Header 的流量切分

1、创建 Canary Ingress，指定 v2 版本的后端服务，且加上一些 annotation，实现仅将带有名为Region 且值为 cd 或 sz 的请求头的请求转发给当前 Canary Ingress，模拟灰度新版本给成都和深圳地域的用户:

v2-ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "Region"
    nginx.ingress.kubernetes.io/canary-by-header-pattern: "cd|sz"
  name: nginx-canary
spec:
  rules:
  - host: canary.example.com
    http:
      paths:
      - backend:
          serviceName: nginx-v2
          servicePort: 80
        path: /

这个与v1.ingress.yaml文件相比，host域名是相同的，serviceName指向 nginx-v2版本。加了一些注解。

执行这个yaml文件。

kubectl apply -f v2-ingress.yaml

 

2、测试访问:
$ curl -H "Host: canary.example.com" -H "Region: cd" http://EXTERNAL-IP #EXTERNAL-IP 替换为 Nginx Ingress 自身对外暴露的 IP

 

[root@k8s-master ingress-nginx]# curl -H "Host: canary.example.com" -H "Region: bj"   http://192.168.24.21
nginx-v1
[root@k8s-master ingress-nginx]# curl -H "Host: canary.example.com" -H "Region: cd"   http://192.168.24.21
nginx-v2

可以看到，只有 header Region 为 cd 或 sz 的请求才由 v2 版本服务响应。

 

三、基于 Cookie 的流量切分：

1、与前面 Header 类似，不过使用 Cookie 就无法自定义 value 了，这里以模拟灰度成都地域用户为例，仅将带有名为 user_from_cd 的 cookie 的请求转发给当前 Canary Ingress 。先删除前面基于 Header 的流量切分的 Canary Ingress，然后创建下面新的 Canary Ingress:

kubectl delete -f v2-ingress.yaml

2、vim v1-cookie.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-cookie: "user_from_cd"
  name: nginx-canary
spec:
  rules:
  - host: canary.example.com
    http:
      paths:
      - backend:
          serviceName: nginx-v2
          servicePort: 80
        path: /
kubectl apply -f v1-cookie.yaml

 

3、验证 

 

[root@k8s-master ingress-nginx]#  curl -s -H "Host: canary.example.com" --cookie "user_from_cd=always" http://192.168.24.21
nginx-v2
[root@k8s-master ingress-nginx]#  curl -s -H "Host: canary.example.com" --cookie "user_from_cd=111" http://192.168.24.21
nginx-v1
[root@k8s-master ingress-nginx]#  curl -s -H "Host: canary.example.com" http://192.168.24.21
nginx-v1

 可以看到，只有 cookie user_from_cd 为 always 的请求才由 v2 版本的服务响应。

 

四、基于服务权重的流量切分


1、基于服务权重的 Canary Ingress 就简单了，直接定义需要导入的流量比例，这里以导入 10% 流量到 v2 版本为例 (如果有，先删除之前的 Canary Ingress):

kubectl delete -f v1-cookie.yaml

 

2、vim v1-weight.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "10"
  name: nginx-canary
spec:
  rules:
  - host: canary.example.com
    http:
      paths:
      - backend:
          serviceName: nginx-v2
          servicePort: 80
        path: /

 

kubectl apply -f v1-weight.yaml

3、验证：

for i in {1..10}; do curl -H "Host: canary.example.com" http://192.168.24.21; done;
[root@k8s-master ingress-nginx]# for i in {1..10}; do curl -H "Host: canary.example.com" http://192.168.24.21; done;
nginx-v1
nginx-v1
nginx-v1
nginx-v1
nginx-v1
nginx-v1
nginx-v2
nginx-v1
nginx-v1
nginx-v1

可以看到，大概只有十分之一的几率由 v2 版本的服务响应，符合 10% 服务权重的设置。

 

posted @ 2022-01-09 13:40  沐雪架构师  阅读(398)  评论(0)  编辑  收藏  举报