kubernetes HPA 示例

web服务

depoly-demoapp-v10.yaml

apiVersion: v1
kind: Namespace
metadata:
    name: hpa-demoapp
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: demoappv10
  name: demoappv10
  namespace: hpa-demoapp
spec:
  #replicas: 1
  selector:
    matchLabels:
      app: demoappv10
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: demoappv10
    spec:
      containers:
      - image: registry.k8s.io/hpa-example
        name: demoapp
        resources:
          limits:
            cpu: 500m
            memory: "256Mi"
          requests:
            cpu: 200m
            memory: "256Mi"

---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: demoappv10-svc
  name: demoappv10-svc
  namespace: hpa-demoapp
spec:
  ports:
  - name: http-8080
    port: 8080
    protocol: TCP
    targetPort: 80
  selector:
    app: demoappv10
  type: ClusterIP

创建资源

# kubectl apply -f depoly-demoapp-v10.yaml

namespace/hpa-demoapp created
deployment.apps/demoappv10 created
service/demoappv10-svc created

查看资源

查看pod

# kubectl get pod -n hpa-demoapp

NAME                         READY   STATUS    RESTARTS   AGE
demoappv10-cdf9995cb-s54n8   1/1     Running   0          43s

查看svc

# kubectl get svc -n hpa-demoapp

NAME             TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
demoappv10-svc   ClusterIP   10.100.1.117   <none>        8080/TCP   10s

访问资源

# curl `kubectl get svc/demoappv10-svc -n hpa-demoapp -o jsonpath="{.spec.clusterIP}"`:8080

OK!

基于单指标自动扩缩

hpa-demoapp.yaml

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: hpa-demoapp
  namespace: hpa-demoapp
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: demoappv10
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 50

创建hpa资源

# kubectl apply -f hpa-demoapp.yaml

horizontalpodautoscaler.autoscaling/hpa-demoapp created

查看hpa资源

# kubectl get hpa -n hpa-demoapp

NAME          REFERENCE               TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
hpa-demoapp   Deployment/demoappv10   0%/50%    1         10        1          47s

请注意当前的 CPU 利用率是 0%

增加负载

访问服务

# while sleep 0.01; do curl `kubectl get svc/demoappv10-svc -n hpa-demoapp -o jsonpath="{.spec.clusterIP}"`:8080;done

OK!OK!OK!OK!OK!......

观察cpu使用情况

# kubectl get hpa/hpa-demoapp -n hpa-demoapp -w

NAME          REFERENCE               TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
hpa-demoapp   Deployment/demoappv10   2%/50%    1         10        1          3m12s
hpa-demoapp   Deployment/demoappv10   84%/50%   1         10        1          3m45s
hpa-demoapp   Deployment/demoappv10   198%/50%   1         10        2          4m
hpa-demoapp   Deployment/demoappv10   176%/50%   1         10        4          4m16s
hpa-demoapp   Deployment/demoappv10   93%/50%    1         10        4          4m31s
hpa-demoapp   Deployment/demoappv10   103%/50%   1         10        4          4m46s
hpa-demoapp   Deployment/demoappv10   98%/50%    1         10        4          5m1s
hpa-demoapp   Deployment/demoappv10   93%/50%    1         10        4          5m16s
hpa-demoapp   Deployment/demoappv10   70%/50%    1         10        4          5m31s
hpa-demoapp   Deployment/demoappv10   60%/50%    1         10        4          5m46s
hpa-demoapp   Deployment/demoappv10   54%/50%    1         10        4          6m1s
hpa-demoapp   Deployment/demoappv10   55%/50%    1         10        4          6m16s
hpa-demoapp   Deployment/demoappv10   53%/50%    1         10        5          6m31s
hpa-demoapp   Deployment/demoappv10   44%/50%    1         10        5          6m46s
hpa-demoapp   Deployment/demoappv10   43%/50%    1         10        5          7m1s
hpa-demoapp   Deployment/demoappv10   41%/50%    1         10        5          7m16s
hpa-demoapp   Deployment/demoappv10   43%/50%    1         10        5          7m31s
hpa-demoapp   Deployment/demoappv10   42%/50%    1         10        5          7m46s

查看pod数量

# kubectl get pod -n hpa-demoapp

NAME                          READY   STATUS    RESTARTS   AGE
demoappv10-57fc7f894c-8mzbf   1/1     Running   0          49s
demoappv10-57fc7f894c-8pmxr   1/1     Running   0          3m4s
demoappv10-57fc7f894c-drv9r   1/1     Running   0          7m5s
demoappv10-57fc7f894c-ls965   1/1     Running   0          3m5s
demoappv10-57fc7f894c-mhblr   1/1     Running   0          3m20s

查看HPA日志

# kubectl describe hpa/hpa-demoapp -n hpa-demoapp

....
  Normal   SuccessfulRescale             6m38s               horizontal-pod-autoscaler  New size: 2; reason: cpu resource utilization (percentage of request) above target
  Normal   SuccessfulRescale             6m23s               horizontal-pod-autoscaler  New size: 4; reason: cpu resource utilization (percentage of request) above target
  Normal   SuccessfulRescale             4m7s                horizontal-pod-autoscaler  New size: 5; reason: cpu resource utilization (percentage of request) above target

停止产生负载

停止访问服务

输入 <Ctrl> + C 来终止负载的产生。

观察cpu使用情况

# kubectl get hpa/hpa-demoapp -n hpa-demoapp -w

...
hpa-demoapp   Deployment/demoappv10   4%/50%     1         10        5          14m
hpa-demoapp   Deployment/demoappv10   2%/50%     1         10        5          15m

查看pod数量

自动扩缩完成副本数量的改变可能需要几分钟的时间。

# kubectl get pod -n hpa-demoapp

NAME                          READY   STATUS    RESTARTS   AGE
demoappv10-57fc7f894c-drv9r   1/1     Running   0          20m

查看HPA日志

# kubectl describe hpa/hpa-demoapp -n hpa-demoapp

....
  Normal   SuccessfulRescale             6m38s               horizontal-pod-autoscaler  New size: 2; reason: cpu resource utilization (percentage of request) above target
  Normal   SuccessfulRescale             6m23s               horizontal-pod-autoscaler  New size: 4; reason: cpu resource utilization (percentage of request) above target
  Normal   SuccessfulRescale             4m7s                horizontal-pod-autoscaler  New size: 5; reason: cpu resource utilization (percentage of request) above target
  Normal   SuccessfulRescale             2m18s               horizontal-pod-autoscaler  New size: 4; reason: All metrics below target
  Normal   SuccessfulRescale             2m2s                horizontal-pod-autoscaler  New size: 1; reason: All metrics below target

基于多项度量指标自动扩缩

CPU 利用率这个度量指标是一个 resource metric（资源度量指标），因为它表示容器上指定资源的百分比。 除 CPU 外，你还可以指定其他资源度量指标。默认情况下，目前唯一支持的其他资源度量指标为内存。 只要 metrics.k8s.io API 存在，这些资源度量指标就是可用的，并且他们不会在不同的 Kubernetes 集群中改变名称。

hpa-v2.yaml

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: php-apache
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: php-apache
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 50
  - type: Resource
    resource:
      name: memory
      target:
        type: AverageValue
        averageValue: 30Mi

HorizontalPodAutoscaler 将会尝试确保每个 Pod 的 CPU 利用率在 50% 以内，可用内存保持在30Mi以上。

基于自定义度量指标自动扩缩

你还可以指定资源度量指标使用绝对数值，而不是百分比，你需要将 target.type 从 Utilization 替换成 AverageValue，同时设置 target.averageValue 而非 target.averageUtilization 的值。
还有两种其他类型的度量指标，他们被认为是 custom metrics（自定义度量指标）： 即 Pod 度量指标和 Object 度量指标。 这些度量指标可能具有特定于集群的名称，并且需要更高级的集群监控设置。

hpa-v2.yaml

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: php-apache
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: php-apache
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 50
  - type: Pods
    pods:
      metric:
        name: packets-per-second
      target:
        type: AverageValue
        averageValue: 1k
  - type: Object
    object:
      metric:
        name: requests-per-second
      describedObject:
        apiVersion: networking.k8s.io/v1
        kind: Ingress
        name: main-route
      target:
        type: Value
        value: 10k

  - type: External
    external:
      metric:
        name: queue_messages_ready
        selector:
          matchLabels:
            queue: "worker_tasks"
      target:
        type: AverageValue
        averageValue: 30

 HorizontalPodAutoscaler 将会尝试确保每个 Pod 的 CPU 利用率在 50% 以内， 每秒能够服务 1000 个数据包请求， 并确保所有在 Ingress 后的 Pod 每秒能够服务的请求总数达到 10000 个。

基于更特别的度量值来扩缩

许多度量流水线允许你通过名称或附加的 标签 来描述度量指标。 对于所有非资源类型度量指标（Pod、Object 和后面将介绍的 External）， 可以额外指定一个标签选择算符。例如，如果你希望收集包含 verb 标签的 http_requests 度量指标，可以按如下所示设置度量指标块，使得扩缩操作仅针对 GET 请求执行：

  - type: Object
    object:
      metric:
        name: http_requests
        selector: {matchLabels: {verb: GET}}

这个选择算符使用与 Kubernetes 标签选择算符相同的语法。 如果名称和标签选择算符匹配到多个系列，监测管道会决定如何将多个系列合并成单个值。 选择算符是可以累加的，它不会选择目标以外的对象（类型为 Pods 的目标 Pod 或者类型为 Object 的目标对象）。

基于与 Kubernetes 对象无关的度量指标执行扩缩

运行在 Kubernetes 上的应用程序可能需要基于与 Kubernetes 集群中的任何对象没有明显关系的度量指标进行自动扩缩， 例如那些描述与任何 Kubernetes 名字空间中的服务都无直接关联的度量指标。

使用外部度量指标时，需要了解你所使用的监控系统，相关的设置与使用自定义指标时类似。 外部度量指标使得你可以使用你的监控系统的任何指标来自动扩缩你的集群。 你需要在 metric 块中提供 name 和 selector，同时将类型由 Object 改为 External。 如果 metricSelector 匹配到多个度量指标，HorizontalPodAutoscaler 将会把它们加和。 外部度量指标同时支持 Value 和 AverageValue 类型，这与 Object 类型的度量指标相同。

例如，如果你的应用程序处理来自主机上消息队列的任务， 为了让每 30 个任务有 1 个工作者实例，你可以将下面的内容添加到 HorizontalPodAutoscaler 的配置中。

- type: External
  external:
    metric:
      name: queue_messages_ready
      selector:
        matchLabels:
          queue: "worker_tasks"
    target:
      type: AverageValue
      averageValue: 30

如果可能，还是推荐定制度量指标而不是外部度量指标，因为这便于让系统管理员加固定制度量指标 API。 而外部度量指标 API 可以允许访问所有的度量指标。 当暴露这些服务时，系统管理员需要仔细考虑这个问题。