dengyouf

导航

< 2025年3月 >
日 一 二 三 四 五 六
23 24 25 26 27 28 1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30 31 1 2 3 4 5

统计

基于ingress-nginx实现灰度发布

基于ingress-nginx实现灰度发布

灰度发布介绍

工作中，我们会经常对应用进行升级发版。在互联网公司尤为频繁，主要为了满足业务的快速发展，我们经常用到的发布方式有滚动更新，蓝绿部署，灰度发布。

  • 滚动更新

依次进行新旧更替，知道旧版本全部被替换

  • 蓝绿部署

两台独立的系统，对外提供服务称为绿系统，带上线的系统称为蓝系统。当蓝系统里面的应用测试完成欧，用户流量切入蓝系统，此时蓝系统将成为绿系统，以前的绿系统可以销毁或用于下次做蓝系统使用。

  • 灰度发布

在同一套集群中存在稳定和灰度两个版本，灰度版本可以限制只针对部分人员使用，带灰度测试验证通过后，可以将灰度版本升级为稳定版本，旧版本可以下线。这种更新方式也称之为金丝雀发布。

Ingress-nginx 实现灰度发布原理

Ingress-nginx 基于 nginx 实现，增加了一组用于实现额外功能的 lua 插件。其通过定义 annotation 来实现不同场景的下的灰度发布。

annotation 说明 备注
nginx.ingress.kubernetes.io/canary-by-header 基于请求头 Request Header 实现流量切分 适用于灰度发布和A/B测试
always -请求会发送到灰度(canary)版本 优先级最高
nginx.ingress.kubernetes.io/canary-by-header-value 匹配请求头的值进行路由，需配合canary-by-header 一起使用
value 匹配 -请求会发送到灰度(canary)版本 优先级次之
nginx.ingress.kubernetes.io/canary-weight 基于服务权重的流量切分,权重范围 0 - 100 适用于蓝绿部署
100 -所有流量发送到灰度(canary)版本
nginx.ingress.kubernetes.io/canary-by-cookie 基于 cookie 的流量切分 适用于灰度发布与 A/B 测试
always -请求会发送到灰度(canary)版本 优先级最低

灰度发布思路

  1. 在集群中运行2个应用版本，一个是stable 版本， 一个是canary 版本
  2. 定义稳定版本 ingress，提供应用的正常访问
  3. 定义 canary 版本，通过设置 annotation 中实现流量切分
  4. 经过一定时间的运行， canary 版本可以正常提供服务后，将其切换为 stable 版本，并将源 stable 版本下线

灰度实现

1. 部署生产 stable 版本

echo "
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: stable
  labels:
    app: stable
spec:
  replicas: 3
  selector:
    matchLabels:
      app: stable
  template:
    metadata:
      labels:
        app: stable
    spec:
      containers:
      - name: myapp-v1
        image: ikubernetes/myapp:v1
        ports:
        - name: http
          protocol: TCP
          containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: stable
  labels:
    app: stable
spec:
  ports:
  - name: http
    port: 80
    targetPort: 80
  selector:
    app: stable
" | tee stable.yaml|kubectl apply -f -

2. 定义Ingress资源访问 stable 版本

echo "---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: stable-ingress
spec:
  rules:
  - host: www.linux.io
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: stable
            port:
              number: 80
  ingressClassName: nginx"|tee stable-ingress.yaml|kubectl apply -f -
# 通过域名访问
➜  ~ curl  www.linux.io
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>

3. 发布新版本 canary 版本

~$ echo "
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: canary
  labels:
    app: canary
spec:
  replicas: 3
  selector:
    matchLabels:
      app: canary
  template:
    metadata:
      labels:
        app: canary
    spec:
      containers:
      - name: myapp-v2
        image: ikubernetes/myapp:v2
        ports:
        - name: http
          protocol: TCP
          containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: canary
  labels:
    app: canary
spec:
  ports:
  - name: http
    port: 80
    targetPort: 80
  selector:
    app: canary
" | tee canary.yaml|kubectl apply -f -

~$ kubectl  get svc canary
NAME     TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
canary   ClusterIP   10.107.157.225   <none>        80/TCP    18s
~$ curl 10.107.157.225
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>

4. 配置 Annotation 规则

4.1 基于权重

echo '---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: canary-ingress
  annotations:
    nginx.ingress.kubernetes.io/canary: "true" # 要开启灰度发布机制，首先需要启用 Canary
    nginx.ingress.kubernetes.io/canary-weight: "30" # 分配30%流量到当前Canary版本
spec:
  rules:
  - host: www.linux.io
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: canary
            port:
              number: 80
  ingressClassName: nginx
  '|tee canary-weight-ingress.yaml|kubectl apply -f -

~$ kubectl  get ingress
NAME             CLASS   HOSTS          ADDRESS          PORTS   AGE
canary-ingress   nginx   www.linux.io                    80      7s
stable-ingress   nginx   www.linux.io   10.105.103.190   80      8m46s

➜  ~ for i in `seq 100`; do curl  www.linux.io; done
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>

4.2 基于Request Header

echo '
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: canary-ingress
  annotations:
    nginx.ingress.kubernetes.io/canary: "true" # 要开启灰度发布机制，首先需要启用 Canary
    nginx.ingress.kubernetes.io/canary-by-header: canary # 基于header的流量切分
    nginx.ingress.kubernetes.io/canary-weight: "30" # 会被忽略，因为配置了 canary-by-headerCanary版本
spec:
  rules:
  - host: www.linux.io
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: canary
            port:
              number: 80
  ingressClassName: nginx
  '|tee canary-request-header-ingress.yaml|kubectl apply -f -

# never : 所有流量发送到stable版本
➜  ~ for i in `seq 10`; do curl   -H "canary: never"  www.linux.io; done
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>

# always : 所有流量发送到canner版本
➜  ~ for i in `seq 10`; do curl   -H "canary: always"  www.linux.io; done
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>

# canary请求头或者请求头不是alway或never : 所有流量按权重方式发送到canner版本
➜  ~ for i in `seq 10`; do curl   -H "canary: other"  www.linux.io; done
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
➜  ~ for i in `seq 10`; do curl     www.linux.io; done
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>

4.3 基于Request Header的value 进行流量切分

echo '
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: canary-ingress
  annotations:
    nginx.ingress.kubernetes.io/canary: "true" # 要开启灰度发布机制，首先需要启用 Canary
    nginx.ingress.kubernetes.io/canary-by-header-value: user-value
    nginx.ingress.kubernetes.io/canary-by-header: canary # 基于header的流量切分
    nginx.ingress.kubernetes.io/canary-weight: "30" # 分配30%流量到当前Canary版本
spec:
  rules:
  - host: www.linux.io
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: canary
            port:
              number: 80
  ingressClassName: nginx
  '|tee canary-request-header-value-ingress.yaml|kubectl apply -f -

# 值匹配全部发往canary版本
➜  ~ for i in `seq 10`; do curl   -H "canary: user-value"   www.linux.io; done
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>

# 值不存在或者不匹配则按权重进行分发
➜  ~ for i in `seq 10`; do curl   -H "canary: always"   www.linux.io; done
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
➜  ~ for i in `seq 10`; do curl   -H "canary: nerver"   www.linux.io; done
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
➜  ~ for i in `seq 10`; do curl    www.linux.io; done
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>

echo '
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: canary-ingress
  annotations:
    nginx.ingress.kubernetes.io/canary: "true" # 要开启灰度发布机制，首先需要启用 Canary
    nginx.ingress.kubernetes.io/canary-by-cookie: "users_from_Beijing" # 基于 cookie
    nginx.ingress.kubernetes.io/canary-weight: "30" # 会被忽略，因为配置了 canary-by-cookie
spec:
  rules:
  - host: www.linux.io
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: canary
            port:
              number: 80
  ingressClassName: nginx
  '|tee canary-request-cookie-ingress.yaml|kubectl apply -f -

# users_from_Beijing=always 全部发往 canary 版本
➜  ~ for i in `seq 10`; do curl  --cookie "users_from_Beijing=always"   www.linux.io; done
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>

# 
➜  ~ for i in `seq 10`; do curl  --cookie  "users_from_Beijing=never"   www.linux.io; done
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
➜  ~ for i in `seq 10`; do curl  -b "users_from_Beijing=other"   www.linux.io; done
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>

posted on   dengyouf  阅读(7)  评论(0)  编辑  收藏  举报

相关博文：
· Ingress-Nginx 快速入门
· Kubernetes 集群安装 MetalLB
· Kubernetes-基于ingress实现灰度发布
· 通过Ingress-nginx实现灰度发布---灰度发布(22)
· Kubernetes 使用 Ingress-nginx 实现灰度发布功能
阅读排行：
· 阿里最新开源QwQ-32B，效果媲美deepseek-r1满血版，部署成本又又又降低了！
· 开源Multi-agent AI智能体框架aevatar.ai，欢迎大家贡献代码
· Manus重磅发布：全球首款通用AI代理技术深度解析与实战指南
· 被坑几百块钱后，我竟然真的恢复了删除的微信聊天记录！
· AI技术革命，工作效率10个最佳AI工具
点击右上角即可分享
微信分享提示