Argo Rollouts TrafficRouting结合Istio进行Canary流量管理基础

Argo Rollouts TrafficRouting概述

流量治理技术实现如下：

1. 按百分比进行流量管理（即 5% 的流量应流向新版本，其余流量流向稳定版本）
2. 基于标头的路由（即将带有特定标头的请求发送到新版本）
3. 镜像流量，其中所有流量都被复制并并行发送到新版本（但响应被忽略）

TrafficRouting 配置

apiVersion: argoproj.io/v1alpha1
kind: Rollout
spec:
  ...
  strategy:
    canary:
      ...
      trafficRouting:
        alb: <Object>
        ambassador: <Object>
        apisix: <Object>
        appMesh: <Object>
        nginx: <Object>
        plugins: <Object>
        smi: <Object>
        traefik: <Object>       
        managedRoutes:               # 管理路由列表
          - name: priority-route-1
          - name: priority-route-2
          - name: priority-route-3
          - name: set-header-1       # 标头路由
          - name: mirror-route       # 镜像路由
        istio: <Object>
      steps:
      - setCanaryScale:              # 设定Canary扩容期间Pod扩增与流量扩增的对应关系    
          weight: 25                 # 明确设定Canary RS的规模为该处指定的比例，但不改变先前设定的流量比例；
      - setMirrorRoute:              # 镜像路由
          name: mirror-route         # 镜像路由的名称。
          percentage: 35             # 要镜像的匹配流量的百分比是多少
          match:                     #  标头路由的匹配规则，如果缺少此规则，则相当于删除该路由。
            - method:
                exact: GET
              path:
                prefix: /
      - setWeight: 20                # 设定Canary版本ReplicSet激活的Pod比例，以及调度至Canary版本的流量比例；
      - setHeaderRoute:              # 标头路由
          name: "set-header-1"       # 标头路由的名称。
          match:                     # 标头匹配规则是 headerName、headerValue 对的数组。
          - headerName: Custom-Header1 # 要匹配的标头名称
            headerValue:
              exact: Mozilla
          - headerName: Custom-Header2 # or Custom-Header2 has a prefix Mozilla
            headerValue:
              prefix: Mozilla
          - headerName: Custom-Header3 # or Custom-Header3 value match regex: Mozilla(.*)
            headerValue:
              regex: Mozilla(.*)  

基于host进行流量分割

配置说明

基于host进行流量分割:
1. 在两个主机名或 Kubernetes 服务之间进行分割：canaryService和stableService。
2. 每个Service代表着一个Host
3. 分别为Canary和Stable的Pod添加rollouts-pod-template-hash标签，其值为相应的RS模板的hash值；
4. 动态调整VS上route中Canary Service和Stable Service的weight进行流量迁移

VirtualService 配置

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: rollout-vsvc
spec:
  gateways:
  - istio-rollout-gateway
  hosts:
  - istio-rollout.dev.argoproj.io
  http:
  - name: primary        # referenced in canary.trafficRouting.istio.virtualService.routes
    route:
    - destination:
        host: stable-svc # referenced in canary.stableService
      weight: 100
    - destination:
        host: canary-svc # referenced in canary.canaryService
      weight: 0

Service 配置

apiVersion: v1
kind: Service
metadata:
  name: canary-svc
spec:
  ports:
  - port: 80
    targetPort: http
    protocol: TCP
    name: http
  selector:
    app: rollouts-demo
    # This selector will be updated with the pod-template-hash of the canary ReplicaSet. e.g.:
    # rollouts-pod-template-hash: 7bf84f9696
---
apiVersion: v1
kind: Service
metadata:
  name: stable-svc
spec:
  ports:
  - port: 80
    targetPort: http
    protocol: TCP
    name: http
  selector:
    app: rollouts-demo
    # This selector will be updated with the pod-template-hash of the stable ReplicaSet. e.g.:
    # rollouts-pod-template-hash: 123746c88d

Rollout 配置

 

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: rollout-example
spec:
  ...
  strategy:
    canary:
      canaryService: canary-svc  # required
      stableService: stable-svc  # required
      trafficRouting:
        istio:
          virtualService:
            name: rollout-vsvc   # required
            routes:
            - primary            # optional if there is a single route in VirtualService, required otherwise
      steps:
      - setWeight: 5
      - pause:
          duration: 10m

基于subset进行流量分割

基于subset进行流量分割:
1. 在两个 Istio DestinationRule 子集之间进行分割：canarySubsetName和stableSubsetName,共用同一个Service。
2. 需要通过DestinationRule将Service的后端的分别隶属于Canary和Stable的Pod划分到不同的subset
3. Pod上的子集划分依赖于一个动态变动的标签进行：canary rollouts-pod-template-hash 和 stable rollouts-pod-template-hash。
4. 分别为Canary和Stable对应的subset上的Pod设定rollouts-pod-template-hash，其值为相应的RS模板的hash值
5. 动态调整VS上route中Canary subset和Stable subset的weight进行流量迁移

VirtualService 配置

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: rollout-vsvc
spec:
  gateways:
  - istio-rollout-gateway
  hosts:
  - istio-rollout.dev.argoproj.io
  http:
  - name: primary       # referenced in canary.trafficRouting.istio.virtualService.routes
    route:
    - destination:
        host: rollout-example
        subset: stable  # referenced in canary.trafficRouting.istio.destinationRule.stableSubsetName
      weight: 100
    - destination:
        host: rollout-example
        subset: canary  # referenced in canary.trafficRouting.istio.destinationRule.canarySubsetName
      weight: 0

DestinationRule 配置 

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: rollout-destrule
spec:
  host: rollout-example
  subsets:
  - name: canary   # referenced in canary.trafficRouting.istio.destinationRule.canarySubsetName
    labels:        # labels will be injected with canary rollouts-pod-template-hash value
      app: rollout-example
  - name: stable   # referenced in canary.trafficRouting.istio.destinationRule.stableSubsetName
    labels:        # labels will be injected with stable rollouts-pod-template-hash value
      app: rollout-example

Service 配置

apiVersion: v1
kind: Service
metadata:
  name: rollout-example
spec:
  ports:
  - port: 80
    targetPort: http
    protocol: TCP
    name: http
  selector:
    app: rollout-example

Rollout 配置

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: rollout-example
spec:
  ...
  strategy:
    canary:
      trafficRouting:
        istio:
          virtualService:
            name: rollout-vsvc        # required
            routes:
            - primary                 # optional if there is a single route in VirtualService, required otherwise
          destinationRule:
            name: rollout-destrule    # required
            canarySubsetName: canary  # required
            stableSubsetName: stable  # required
      steps:
      - setWeight: 5
      - pause:
          duration: 10m

基于TCP进行流量分割

VirtualService 配置

VirtualService 必须包含 TCP 路由以及 Rollout 中引用的匹配端口。

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: rollout-vsvc
spec:
  gateways:
    - istio-rollout-gateway
  hosts:
    - istio-rollout.dev.argoproj.io
  tcp:
    - match:
        - port: 3000
      route:
        - destination:
            host: stable-svc # referenced in canary.stableService
          weight: 100
        - destination:
            host: canary-svc # referenced in canary.canaryService
          weight: 0

Rollout 配置

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: rollout-example
spec:
  ...
  strategy:
    canary:
      canaryService: canary-svc  # required
      stableService: stable-svc  # required
      trafficRouting:
        istio:
          virtualService:
            name: rollout-vsvc   # required
            tcpRoutes:
              # Below fields are optional but if defined, they should match exactly with at least one of the TCP route match rules in your VirtualService
              - port: 3000 # Only required if you want to match any rule in your VirtualService which contains this port
      steps:
      - setWeight: 5
      - pause:
          duration: 10m

与 GitOps 集成

Argo CD 与 Argo Rollouts 一起使用时，因Argo Rollouts 会动态调整vs会导致权重的瞬时摆动。解决办法如下：

Application 忽略 VirtualService 中的差异 

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: guestbook
spec:
  ignoreDifferences:
  - group: networking.istio.io
    kind: VirtualService
    jsonPointers:
    - /spec/http/0

配置 Application 仅应用 OutOfSync 资源 

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: guestbook
spec:
  syncPolicy:
    syncOptions:
    - ApplyOutOfSyncOnly=true

默认情况下，当 Argo CD 同步application时，它会针对 git 中作为application一部分的所有资源运行 kubectl apply。 ApplyOutOfSyncOnly=true 同步选项指示 Argo CD 跳过应用它已认为已同步的资源，而仅应用不同步的资源。当与ignoreDifferences 功能结合使用时，此选项提供了一种管理Argo CD 和Argo Rollouts 之间VirtualService 所需状态冲突的方法。

参考文档

https://argoproj.github.io/argo-rollouts/features/traffic-management/istio/