k8s-容器资源需求、资源限制及HeapSter-二十二
一、容器资源需求、资源限制
资源需求、资源限制：指的是cpu、内存等资源；
资源需求、资源限制的两个关键字：
- request：需求，最低保障，在调度时，这个节点必须要满足request需求的资源大小；
- limits：限制、硬限制。这个限制容器无论怎么运行都不会超过limits的值；
CPU：k8s的一个cpu对应一颗宿主机逻辑cpu。一个逻辑cpu还可以划分为1000个毫核（millcores）。所以1cpu=1000m；500m=0.5个CPU，0.5m相当于二分之一的核心；
内存的计量单位：E、P、T、G、M、K
[root@master ~]# kubectl explain pods.spec.containers.resources [root@master ~]# kubectl explain pods.spec.containers.resources.requests [root@master ~]# kubectl explain pods.spec.containers.resources.limits
用法参考：https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/
[root@master metrics]# pwd /root/manifests/metrics [root@master metrics]# vim pod-demo.yaml apiVersion: v1 kind: Pod metadata: name: pod-demo labels: app: myapp tier: frontend spec: containers: - name: myapp image: ikubernetes/stress-ng command: ["/usr/bin/stress-ng", "-c 1", "--metrics-brief"] #-c 1表示启动一个子进程对cpu做压测.默认stress-ng的一个子进程使用256M内存 resources: requests: cpu: "200m" memory: "128Mi" limits: cpu: "500m" memory: "512Mi" #创建pod [root@master metrics]# kubectl apply -f pod-demo.yaml pod/pod-demo created [root@master metrics]# kubectl get pods NAME READY STATUS RESTARTS AGE pod-demo 1/1 Running 0 6s [root@master metrics]# kubectl exec pod-demo -- top Mem: 1378192K used, 487116K free, 12540K shrd, 2108K buff, 818184K cached CPU: 26% usr 1% sys 0% nic 71% idle 0% io 0% irq 0% sirq Load average: 0.78 0.96 0.50 2/479 11 PID PPID USER STAT VSZ %VSZ CPU %CPU COMMAND 6 1 root R 6884 0% 1 26% {stress-ng-cpu} /usr/bin/stress-ng 7 0 root R 1504 0% 0 0% top 1 0 root S 6244 0% 1 0% /usr/bin/stress-ng -c 1 --metrics-
我们对容器分配了资源限制后，k8s会自动分配一个QoS，叫服务质量，通过kubectl describe pods pod_name可以查看这个字段；
[root@master metrics]# kubectl describe pods pod-demo |grep QoS QoS Class: Burstable
QoS可以分为三类(根据资源设置，自动归类)：
- Guranteed：表示每个容器的cpu和内存资源设置了相同的requests和limits值，即cpu.requests=cpu.limits和memory.requests=memory.limits，Guranteed会确保这类pod有最高的优先级，会被优先运行的，即使节点上的资源不够用；
- Burstable:表示pod中至少有一个容器设置了cpu或内存资源的requests属性，可能没有定义limits属性，那么这类pod具有中等优先级；
- BestEffort：指没有任何一个容器设置了requests或者limits属性，那么这类pod是最低优先级。当这类pod的资源不够用时，BestEffort中的容器会被优先终止，以便腾出资源来，给另外两类pod中的容器正常运行；
二、HeapSter
1、介绍
HeapSter的作用是收集个节点pod的资源使用情况，然后以图形界面展示给用户。
kubelet中的cAdvisor负责收集每个节点上的资源使用情况，然后把信息存储HeapSter中，HeapSter再把数据持久化的存储在数据库InfluxDB中。然后我们再通过非常优秀的Grafana来图形化展示；
一般我们监控的指标包括k8s集群的系统指标、容器指标和应用指标。
默认InfluxDB使用的是存储卷是emptyDir，容器一关数据就没了，所以我们生产要换成glusterfs等存储卷才行。
2、部署influxdb：
InfluxDB github：https://github.com/kubernetes-retired/heapster
在node节点上先拉取镜像：
#node01 [root@node01 ~]# docker pull fishchen/heapster-influxdb-amd64:v1.5.2 #node02 [root@node02 ~]# docker pull fishchen/heapster-influxdb-amd64:v1.5.2
在master节点上，拉取yaml文件，并修改、执行：
[root@master metrics]# wget https://raw.githubusercontent.com/kubernetes-retired/heapster/master/deploy/kube-config/influxdb/influxdb.yaml [root@master metrics]# vim influxdb.yaml apiVersion: apps/v1 #此处不修改也可以，如果改成apps/v1，要加下面 selector那几行 kind: Deployment metadata: name: monitoring-influxdb namespace: kube-system spec: replicas: 1 selector: #加此行 matchLabels: #加此行 task: monitoring #加此行 k8s-app: influxdb #加此行 template: metadata: labels: task: monitoring k8s-app: influxdb spec: containers: - name: influxdb image: fishchen/heapster-influxdb-amd64:v1.5.2 #修改此处镜像地址 volumeMounts: - mountPath: /data name: influxdb-storage volumes: - name: influxdb-storage emptyDir: {} --- apiVersion: v1 kind: Service metadata: labels: task: monitoring # For use as a Cluster add-on (https://github.com/kubernetes/kubernetes/tree/master/cluster/addons) # If you are NOT using this as an addon, you should comment out this line. kubernetes.io/cluster-service: 'true' kubernetes.io/name: monitoring-influxdb name: monitoring-influxdb namespace: kube-system spec: ports: - port: 8086 targetPort: 8086 selector: k8s-app: influxdb #创建资源 [root@master metrics]# kubectl apply -f influxdb.yaml deployment.apps/monitoring-influxdb created service/monitoring-influxdb created #查看 [root@master metrics]# kubectl get pods -n kube-system |grep influxdb monitoring-influxdb-5899b7fff9-2r58w 1/1 Running 0 6m59s [root@master metrics]# kubectl get svc -n kube-system |grep influxdb monitoring-influxdb ClusterIP 10.101.242.217 <none> 8086/TCP 7m6s
3、部署rbac
下面我们开始部署heapster，但heapster依赖rbac，所以我们先部署rbac：
[root@master metrics]# wget https://raw.githubusercontent.com/kubernetes-retired/heapster/master/deploy/kube-config/rbac/heapster-rbac.yaml [root@master metrics]# kubectl apply -f heapster-rbac.yaml clusterrolebinding.rbac.authorization.k8s.io/heapster created
4、部署heapster
#node01拉取镜像 [root@node01 ~]# docker pull rancher/heapster-amd64:v1.5.4 #node02拉取镜像 [root@node02 ~]# docker pull rancher/heapster-amd64:v1.5.4 #master拉取yaml文件 [root@master metrics]# wget https://raw.githubusercontent.com/kubernetes-retired/heapster/master/deploy/kube-config/influxdb/heapster.yaml [root@master metrics]# vim heapster.yaml apiVersion: v1 kind: ServiceAccount metadata: name: heapster namespace: kube-system --- apiVersion: apps/v1 kind: Deployment metadata: name: heapster namespace: kube-system spec: replicas: 1 selector: matchLabels: task: monitoring k8s-app: heapster template: metadata: labels: task: monitoring k8s-app: heapster spec: serviceAccountName: heapster containers: - name: heapster image: rancher/heapster-amd64:v1.5.4 #修改此处镜像地址 imagePullPolicy: IfNotPresent command: - /heapster - --source=kubernetes:https://kubernetes.default - --sink=influxdb:http://monitoring-influxdb.kube-system.svc:8086 --- apiVersion: v1 kind: Service metadata: labels: task: monitoring # For use as a Cluster add-on (https://github.com/kubernetes/kubernetes/tree/master/cluster/addons) # If you are NOT using this as an addon, you should comment out this line. kubernetes.io/cluster-service: 'true' kubernetes.io/name: Heapster name: heapster namespace: kube-system spec: ports: - port: 80 targetPort: 8082 type: NodePort #我添加了此行， selector: k8s-app: heapster #创建 [root@master metrics]# kubectl apply -f heapster.yaml serviceaccount/heapster created deployment.apps/heapster created #查看 [root@master metrics]# kubectl get pods -n kube-system |grep heapster- heapster-7c8f7dc8cb-kph29 1/1 Running 0 3m55s [root@master metrics]# [root@master metrics]# kubectl get svc -n kube-system |grep heapster heapster NodePort 10.111.93.84 <none> 80:31410/TCP 4m16s #由于用了NodePort，所以pod端口映射到了节点31410端口上 #查看pod日志 [root@master metrics]# kubectl logs heapster-7c8f7dc8cb-kph29 -n kube-system
5、部署Grafana
#node01拉取镜像 [root@node01 ~]# docker pull angelnu/heapster-grafana:v5.0.4 #node02拉取镜像 [root@node02 ~]# docker pull angelnu/heapster-grafana:v5.0.4 #master拉取yaml文件 [root@master metrics]# wget https://raw.githubusercontent.com/kubernetes-retired/heapster/master/deploy/kube-config/influxdb/grafana.yaml #编辑yaml文件 [root@master metrics]# vim grafana.yaml apiVersion: apps/v1 kind: Deployment metadata: name: monitoring-grafana namespace: kube-system spec: replicas: 1 selector: matchLabels: task: monitoring k8s-app: grafana template: metadata: labels: task: monitoring k8s-app: grafana spec: containers: - name: grafana image: angelnu/heapster-grafana:v5.0.4 #修改镜像地址 ports: - containerPort: 3000 protocol: TCP volumeMounts: - mountPath: /etc/ssl/certs name: ca-certificates readOnly: true - mountPath: /var name: grafana-storage env: - name: INFLUXDB_HOST value: monitoring-influxdb - name: GF_SERVER_HTTP_PORT value: "3000" # The following env variables are required to make Grafana accessible via # the kubernetes api-server proxy. On production clusters, we recommend # removing these env variables, setup auth for grafana, and expose the grafana # service using a LoadBalancer or a public IP. - name: GF_AUTH_BASIC_ENABLED value: "false" - name: GF_AUTH_ANONYMOUS_ENABLED value: "true" - name: GF_AUTH_ANONYMOUS_ORG_ROLE value: Admin - name: GF_SERVER_ROOT_URL # If you're only using the API Server proxy, set this value instead: # value: /api/v1/namespaces/kube-system/services/monitoring-grafana/proxy value: / volumes: - name: ca-certificates hostPath: path: /etc/ssl/certs - name: grafana-storage emptyDir: {} --- apiVersion: v1 kind: Service metadata: labels: # For use as a Cluster add-on (https://github.com/kubernetes/kubernetes/tree/master/cluster/addons) # If you are NOT using this as an addon, you should comment out this line. kubernetes.io/cluster-service: 'true' kubernetes.io/name: monitoring-grafana name: monitoring-grafana namespace: kube-system spec: # In a production setup, we recommend accessing Grafana through an external Loadbalancer # or through a public IP. # type: LoadBalancer # You could also use NodePort to expose the service at a randomly-generated port # type: NodePort ports: - port: 80 targetPort: 3000 type: NodePort #为了能在集群外部访问Grafana，所以我们需要定义NodePort selector: k8s-app: grafana #创建 [root@master metrics]# kubectl apply -f grafana.yaml deployment.apps/monitoring-grafana created service/monitoring-grafana created #查看 [root@master metrics]# kubectl get pods -n kube-system |grep grafana monitoring-grafana-84786758cc-7txwr 1/1 Running 0 3m47s [root@master metrics]# kubectl get svc -n kube-system |grep grafana monitoring-grafana NodePort 10.102.42.86 <none> 80:31404/TCP 3m55s #可见pod的端口映射到了node上的31404端口上
pod的端口已经映射到了node上的31404端口上；
此时，在集群外部，用浏览器访问：http://ip:31404
如下图：
但是如何使用，可能还需要进一步学习influxbd、grafana等；
最后，HeapSter可能快被废除了…
新型的监控系统比如有：Prometheus(普罗米修斯)
