kubernetes的弹性伸缩

       通过手工执行kubectl scale命令，可以实现pod扩容或缩容。但是不符合对kubernetes的定位目标——自动化、智能化。分布式系统要能够根据当前负载的变化情况自动触发水平扩展或者缩容行为，因为这一过程可能是频繁发生、不可预料的，所以手动控制的方式不现实。

      Horizontal Pod Autoscaler（pod横向自动扩容，简称HPA）。HPA与RC、Deployment一样，也属于一种kubernetes资源对象。通过跟踪分析RC控制的所有目标pod的负载变化情况，来确定是否需要针对性的调整目标Pod的副本数，这是HPA的实现原理。HPA用于实现基于CPU使用率进行自动pod扩容和缩容的功能。HPA控制器基于Master的kube-controller-manager服务启动参数--horizontal-pod-autoscaler-sync-period定义的时长（默认值为30s），周期性的监测目标pod的cpu使用率，并在满足条件时对ReplicationController或者Deployment中的pod的副本数量进行调整，以符合用户定义的平均pod cpu使用率。pod cpu使用率来自于heapster组件。

    创建HPA时可以使用kubectl autoscale命令进行快速创建或者使用yaml配置文件进行创建。在创建HPA之前，需要已经存在一个deployment/RC对象，并且该Deployment/RC中的pod必须定义resources.requests.cpu的资源请求值，如果不设置该值，则heapster将无法采集到该pod的cpu使用情况，会导致HPA无法正常工作。

     为一个RC设置HPA，然后使用一个客户端对其进行压力测试。

[root@kub_master ~]# cd k8s/
[root@kub_master k8s]# mkdir hpa
[root@kub_master k8s]# cd hpa
[root@kub_master hpa]# vim nginx-rc.yaml 
[root@kub_master hpa]# cat nginx-rc.yaml 
apiVersion: v1
kind: ReplicationController
metadata:
  name: myweb
spec:
  replicas: 3
  selector:
    app: myweb
  template:
    metadata:
      labels:
        app: myweb
    spec:
      containers:
      - name: myweb
        image: 192.168.0.212:5000/nginx:1.15
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 80
        resources:
          limits:
            cpu: 100m
            memory: 50Mi
          requests:
            cpu: 100m
            memory: 50Mi
[root@kub_master hpa]# kubectl create -f nginx-rc.yaml 
replicationcontroller "myweb" created
[root@kub_master hpa]# kubectl get all -o wide
NAME       DESIRED   CURRENT   READY     AGE       CONTAINER(S)   IMAGE(S)                        SELECTOR
rc/myweb   3         3         3         59s       myweb          192.168.0.212:5000/nginx:1.15   app=myweb
rc/nginx   2         2         2         2d        nginx          192.168.0.212:5000/nginx:1.15   app=nginx,version=1.15

NAME             CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE       SELECTOR
svc/kubernetes   192.168.0.1   <none>        443/TCP   8d        <none>

NAME             READY     STATUS    RESTARTS   AGE       IP            NODE
po/myweb-2jmrf   1/1       Running   0          59s       172.16.81.6   192.168.0.212
po/myweb-lq299   1/1       Running   0          59s       172.16.46.3   192.168.0.184
po/myweb-r3446   1/1       Running   0          59s       172.16.66.3   192.168.0.208
po/nginx-bp61w   1/1       Running   0          1d        172.16.81.3   192.168.0.212
po/nginx-sg498   1/1       Running   0          1d        172.16.66.2   192.168.0.208

    接下来为RC“myweb”创建一个HPA控制器，在1和8之间调整pod的副本数量，以使得平均pod cpu使用率维持在10%

    使用kubectl autoscale命令进行创建

[root@kub_master hpa]# kubectl autoscale replicationcontroller myweb --max=8 --min=1 --cpu-percent=10
replicationcontroller "myweb" autoscaled
[root@kub_master hpa]# kubectl get all -o wide
NAME        REFERENCE                     TARGET    CURRENT   MINPODS   MAXPODS   AGE
hpa/myweb   ReplicationController/myweb   10%       0%        1         8         13s

NAME       DESIRED   CURRENT   READY     AGE       CONTAINER(S)   IMAGE(S)                        SELECTOR
rc/myweb   1         1         1         10m       myweb          192.168.0.212:5000/nginx:1.15   app=myweb
rc/nginx   2         2         2         2d        nginx          192.168.0.212:5000/nginx:1.15   app=nginx,version=1.15

NAME             CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE       SELECTOR
svc/kubernetes   192.168.0.1   <none>        443/TCP   8d        <none>

NAME             READY     STATUS    RESTARTS   AGE       IP            NODE
po/myweb-lq299   1/1       Running   0          10m       172.16.46.3   192.168.0.184
po/nginx-bp61w   1/1       Running   0          1d        172.16.81.3   192.168.0.212
po/nginx-sg498   1/1       Running   0          1d        172.16.66.2   192.168.0.208

或者通过yaml配置文件创建HPA，需要在scaleTargetRef字段指定需要管理的Deployment/RC的名字，然后设置minReplicas、maxReplicas和targetCPUUtilizationPercentage参数。

[root@kub_master hpa]# vim hpa-myweb.yaml
[root@kub_master hpa]# cat hpa-myweb.yaml 
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  name: myweb
spec:
  scaleTargetRef:
     apiVersion: apps/v1beta1
     kind: ReplicationController
     name: myweb
  minReplicas: 1
  maxReplicas: 8
  targetCPUUtilizationPercentage: 10

[root@kub_master hpa]# kubectl create -f hpa-myweb.yaml

[root@kub_master hpa]# kubectl get hpa -o wide
NAME      REFERENCE                     TARGET    CURRENT   MINPODS   MAXPODS   AGE
myweb     ReplicationController/myweb   10%       0%        1         8         9m

压力测试：

#压力测试工具命令
[root@kub_master hpa]# yum install httpd-tools -y

[root@kub_master hpa]# kubectl get all -o wide
NAME        REFERENCE                     TARGET    CURRENT   MINPODS   MAXPODS   AGE
hpa/myweb   ReplicationController/myweb   10%       0%        1         8         41m

NAME       DESIRED   CURRENT   READY     AGE       CONTAINER(S)   IMAGE(S)                        SELECTOR
rc/myweb   1         1         1         51m       myweb          192.168.0.212:5000/nginx:1.15   app=myweb
rc/nginx   2         2         2         2d        nginx          192.168.0.212:5000/nginx:1.15   app=nginx,version=1.15

NAME             CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE       SELECTOR
svc/kubernetes   192.168.0.1   <none>        443/TCP   9d        <none>

NAME             READY     STATUS    RESTARTS   AGE       IP            NODE
po/myweb-lq299   1/1       Running   0          51m       172.16.46.3   192.168.0.184
po/nginx-bp61w   1/1       Running   0          1d        172.16.81.3   192.168.0.212
po/nginx-sg498   1/1       Running   0          1d        172.16.66.2   192.168.0.208
#压测命令
[root@kub_master hpa]# ab -n 500000 -c 100 http://172.16.46.3/index.html
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 172.16.46.3 (be patient)
Completed 50000 requests
Completed 100000 requests

#扩容的

[root@kub_master hpa]# kubectl get all -o wide
NAME        REFERENCE                     TARGET    CURRENT   MINPODS   MAXPODS   AGE
hpa/myweb   ReplicationController/myweb   10%       74%       1         8         42m

NAME       DESIRED   CURRENT   READY     AGE       CONTAINER(S)   IMAGE(S)                        SELECTOR
rc/myweb   4         4         4         53m       myweb          192.168.0.212:5000/nginx:1.15   app=myweb
rc/nginx   2         2         2         2d        nginx          192.168.0.212:5000/nginx:1.15   app=nginx,version=1.15

NAME             CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE       SELECTOR
svc/kubernetes   192.168.0.1   <none>        443/TCP   9d        <none>

NAME             READY     STATUS    RESTARTS   AGE       IP            NODE
po/myweb-4292v   1/1       Running   0          11s       172.16.81.7   192.168.0.212
po/myweb-bw6sj   1/1       Running   0          11s       172.16.66.3   192.168.0.208
po/myweb-lq299   1/1       Running   0          53m       172.16.46.3   192.168.0.184
po/myweb-x6x5g   1/1       Running   0          11s       172.16.81.6   192.168.0.212
po/nginx-bp61w   1/1       Running   0          1d        172.16.81.3   192.168.0.212
po/nginx-sg498   1/1       Running   0          1d        172.16.66.2   192.168.0.208

    可以看到HPA已经根据Pod的CPU使用率的提高对RC进行了自动扩容。

#缩容

[root@kub_master hpa]# kubectl get all -o wide
NAME        REFERENCE                     TARGET    CURRENT   MINPODS   MAXPODS   AGE
hpa/myweb   ReplicationController/myweb   10%       0%        1         8         47m

NAME       DESIRED   CURRENT   READY     AGE       CONTAINER(S)   IMAGE(S)                        SELECTOR
rc/myweb   1         1         1         58m       myweb          192.168.0.212:5000/nginx:1.15   app=myweb
rc/nginx   2         2         2         2d        nginx          192.168.0.212:5000/nginx:1.15   app=nginx,version=1.15

NAME             CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE       SELECTOR
svc/kubernetes   192.168.0.1   <none>        443/TCP   9d        <none>

NAME             READY     STATUS    RESTARTS   AGE       IP            NODE
po/myweb-lq299   1/1       Running   0          58m       172.16.46.3   192.168.0.184
po/nginx-bp61w   1/1       Running   0          1d        172.16.81.3   192.168.0.212
po/nginx-sg498   1/1       Running   0          1d        172.16.66.2   192.168.0.208

#压测的整个过程
[root@kub_master hpa]# ab -n 500000 -c 100 http://172.16.46.3/index.html
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 172.16.46.3 (be patient)
Completed 50000 requests
Completed 100000 requests
Completed 150000 requests
Completed 200000 requests
Completed 250000 requests
Completed 300000 requests
Completed 350000 requests
Completed 400000 requests
Completed 450000 requests
Completed 500000 requests
Finished 500000 requests


Server Software:        nginx/1.15.5
Server Hostname:        172.16.46.3
Server Port:            80

Document Path:          /index.html
Document Length:        612 bytes

Concurrency Level:      100
Time taken for tests:   177.998 seconds
Complete requests:      500000
Failed requests:        0
Write errors:           0
Total transferred:      422500000 bytes
HTML transferred:       306000000 bytes
Requests per second:    2809.02 [#/sec] (mean)
Time per request:       35.600 [ms] (mean)
Time per request:       0.356 [ms] (mean, across all concurrent requests)
Transfer rate:          2317.99 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    9  95.8      1    7016
Processing:     0   26  65.5      3    5097
Waiting:        0   25  63.8      3    5097
Total:          1   35 119.1      5    7086

Percentage of the requests served within a certain time (ms)
  50%      5
  66%      6
  75%     13
  80%     73
  90%     82
  95%     88
  98%    210
  99%    409
 100%   7086 (longest request)

   可以看到HPA根据Pod CPU使用率的降低对副本数量进行了缩容操作，pod副本数量变成了1个。

    当前的HPA还只支持将CPU使用率作为pod副本扩容和缩容的触发条件，在将来的版本中，将会支持应用自定义指标作为触发条件。