基于cpu和内存进行pod扩容,创建hpa

创建镜像

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
mkdir php
cd php
touch dockerfile
touch index.php
 
vim dockerfile
FROM php:5-apache
ADD index.php /var/www/html/index.php
RUN chmod a+rx index.php
 
 
vim index.php
<?php
$x = 0.0001;
for ($i = 0; $i <= 1000000;$i++) {
$x += sqrt($x);
}
echo "OK!";
?>
 
docker build -t k8s.gcr.io/hpa-example:v1 .
 
docker save -o hpa-example.tar.gz k8s.gcr.io/hpa-example:v1

 

创建deployment,service

 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
vim php-apache.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: php-apache
spec:
selector:
matchLabels:
run: php-apache
replicas: 1
template:
metadata:
labels:
run: php-apache
spec:
containers:
- name: php-apache
image: k8s.gcr.io/hpa-example:v1
ports:
- containerPort: 80
resources:
limits:
cpu: 500m
requests:
cpu: 200m
 
---
apiVersion: v1
kind: Service
metadata:
name: php-apache
labels:
run: php-apache
spec:
ports:
- port: 80
selector:
run: php-apache
 
 
kubectl apply -f php-apache.yaml

  

创建hpa,基于cpu指标


1)让副本数维持在 1-10 个之间(这里副本数指的是通过 deployment 部署的 pod 的副本数)
2)将所有 Pod 的平均 CPU 使用率维持在 50%(通过 kubectl run 运行的每个 pod 如果是 200
毫核,这意味着平均 CPU 利用率为 100 毫核)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
创建hpa
 
kubectl autoscale deployment php-apache --cpu-percent=50 --min=1 --max=10
 
kubectl autoscale deployment php-apache (php-apache 表示 deployment 的名字) --
cpu-percent=50(表示 cpu 使用率不超过 50%) --min=1(最少一个 pod)
--max=10(最多 10 个 pod)
 
[root@master php]# kubectl autoscale deployment php-apache --cpu-percent=50 --min=1 --max=10
 
horizontalpodautoscaler.autoscaling/php-apache autoscaled
You have new mail in /var/spool/mail/root
[root@master php]#
[root@master php]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
php-apache Deployment/php-apache <unknown>/50% 1 10 0 6s
 
  
 
压力测试
 
busybox
kubectl run v1 -it --image=busybox /bin/sh
while true; do wget -q -O- http://php-apache.default.svc.cluster.local; done
 
 
kubectl run v2 -it --image=busybox /bin/sh
while true; do wget -q -O- http://php-apache.default.svc.cluster.local; done
 
 
[root@master php]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
php-apache Deployment/php-apache <unknown>/50% 1 10 0 6s
[root@master php]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
php-apache Deployment/php-apache 0%/50% 1 10 1 2m8s
You have new mail in /var/spool/mail/root
[root@master php]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
php-apache Deployment/php-apache 0%/50% 1 10 1 2m13s
[root@master php]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
php-apache Deployment/php-apache 0%/50% 1 10 1 2m15s
[root@master php]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
php-apache Deployment/php-apache 179%/50% 1 10 4 2m53s
[root@master php]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
php-apache Deployment/php-apache 179%/50% 1 10 4 2m54s
[root@master php]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
php-apache Deployment/php-apache 82%/50% 1 10 5 3m8s
You have new mail in /var/spool/mail/root
[root@master php]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
php-apache Deployment/php-apache 82%/50% 1 10 5 3m11s
 
  
 
  
 
停止一个请求,还有一个请求存在,发现pod副本的数量没有降下来
 
kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
php-apache-775dff594-48dp8 1/1 Terminating 0 7m8s 192.168.166.147 node1 <none> <none>
php-apache-775dff594-6q4tv 1/1 Running 0 6m53s 192.168.166.149 node1 <none> <none>
php-apache-775dff594-htfs9 1/1 Terminating 0 6m38s 192.168.166.150 node1 <none> <none>
php-apache-775dff594-jct7v 1/1 Running 0 7m8s 192.168.135.17 node3 <none> <none>
php-apache-775dff594-jnm54 1/1 Running 0 13m 192.168.166.146 node1 <none> <none>
php-apache-775dff594-mth8j 1/1 Running 0 7m8s 192.168.135.16 node3 <none> <none>
php-apache-775dff594-qjp9k 1/1 Running 0 6m38s 192.168.135.18 node3 <none> <none>
php-apache-775dff594-sbkhq 1/1 Terminating 0 6m7s 192.168.166.151 node1 <none> <none>
php-apache-775dff594-wxv4k 1/1 Running 0 6m8s 192.168.135.19 node3 <none> <none>
v1 1/1 Running 0 8m19s 192.168.135.15 node3 <none> <none>
 
发现有的pod在逐渐的删除

  

 


基于内存指标进行pod的扩容

 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-hpa
spec:
selector:
matchLabels:
app: nginx
replicas: 1
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.7.9
ports:
- containerPort: 80
name: http
protocol: TCP
resources:
requests:
cpu: 0.01
memory: 25Mi
limits:
cpu: 0.05
memory: 60Mi
---
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
selector:
app: nginx
type: NodePort
ports:
- name: http
protocol: TCP
port: 80
targetPort: 80
nodePort: 30080
nginx 的 pod 里需要有如下字段,否则 hpa 会采集不到内存指标
resources:
requests:
cpu: 0.01
memory: 25Mi
limits:
cpu: 0.05
memory: 60Mi
 
 
创建hpa
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: nginx-hpa
spec:
maxReplicas: 10
minReplicas: 1
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: nginx-hpa
metrics:
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 60
 
[root@master mem]# kubectl api-versions|grep auto
autoscaling/v1
autoscaling/v2
[root@master mem]# vim mem-hpa.yaml
[root@master mem]# kubectl apply -f mem-hpa.yaml
Error from server (BadRequest): error when creating "mem-hpa.yaml": HorizontalPodAutoscaler in version "v2" cannot be handled as a HorizontalPodAutoscaler: strict decoding error: unknown field "spec.metrics[0].resource.targetAverageUtilization"
 
The HorizontalPodAutoscaler "nginx-hpa" is invalid:
* spec.metrics[0].resource.target.type: Required value: must specify a metric target type
* spec.metrics[0].resource.target.type: Invalid value: "": must be either Utilization, Value, or AverageValue
 
  
 
 
HorizontalPodAutoscaler in version "v2" cannot be handled as a HorizontalPodAutoscaler
 
averageUtilization:60
指定 averageUtilization 和所有 pod 的目标平均内存利用率,表示为请求内存的百分比。目标 pod 必须配置内存请求
 
[root@master mem]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx-hpa Deployment/nginx-hpa 27%/60% 1 10 1 58s
 
kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-hpa-7d44f846d6-fhgkp 1/1 Running 0 11m
 
 
登录到上面通过 pod 创建的 nginx,并生成一个文件,增加内存
 
kubectl exec -it nginx-hpa-7d44f846d6-fhgkp -- /bin/sh
dd if=/dev/zero of=/tmp/a
 
[root@master php]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx-hpa Deployment/nginx-hpa 91%/60% 1 10 1 3m2s
[root@master php]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx-hpa Deployment/nginx-hpa 181%/60% 1 10 2 3m4s
 
扩展:查看 v2 版本的 hpa 如何定义?
[root@master mem]# kubectl get hpa.v2beta2.autoscaling -o yaml > 1.yaml
error: the server doesn't have a resource type "hpa"
[root@master mem]# kubectl get hpa.v2.autoscaling -o yaml > 1.yaml