K8S安装prometheus+kube-state-metrics+grafana
k8s安装Prometheus+Grafana： 1.在kubernetest集群中创建namespace： [root@master k8s-promethus]# cat namespace.yaml apiVersion: v1 kind: Namespace metadata: name: ns-monitor labels: name: ns-monitor [root@master k8s-promethus]# kubectl apply -f namespace.yaml 2.安装node-exporter: [root@master k8s-promethus]# cat node-exporter.yaml kind: DaemonSet apiVersion: apps/v1 metadata: labels: app: node-exporter name: node-exporter namespace: ns-monitor spec: revisionHistoryLimit: 10 selector: matchLabels: app: node-exporter template: metadata: labels: app: node-exporter spec: containers: - name: node-exporter image: prom/node-exporter:v0.16.0 ports: - containerPort: 9100 protocol: TCP name: http hostNetwork: true hostPID: true tolerations: - effect: NoSchedule operator: Exists --- kind: Service apiVersion: v1 metadata: labels: app: node-exporter name: node-exporter-service namespace: ns-monitor spec: ports: - name: http port: 9100 protocol: TCP type: NodePort selector: app: node-exporter [root@master k8s-promethus]# kubectl apply -f node-exporter.yaml
[root@master k8s-promethus]# kubectl get pod -n ns-monitor -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES node-exporter-cl6d5 1/1 Running 15 3h15m 192.168.100.64 test <none> <none> node-exporter-gwwnj 1/1 Running 0 3h15m 192.168.100.201 node1 <none> <none> node-exporter-hbglm 1/1 Running 0 3h15m 192.168.100.200 master <none> <none> node-exporter-kwsfv 1/1 Running 0 3h15m 192.168.100.202 node2 <none> <none> [root@master k8s-promethus]# kubectl get svc -n ns-monitor NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE node-exporter-service NodePort 10.99.128.173 <none> 9100:30372/TCP 3h17m
访问测试：
说明：node-exporter已经运行在成功。
3.部署Prometheus ： 注意：prometheus.yaml 中包含rbac认证、ConfigMap等
[root@master k8s-promethus]# kubectl apply -f prometheus.yaml
clusterrole.rbac.authorization.k8s.io/prometheus unchanged serviceaccount/prometheus unchanged clusterrolebinding.rbac.authorization.k8s.io/prometheus unchanged configmap/prometheus-conf unchanged configmap/prometheus-rules unchanged persistentvolume/prometheus-data-pv unchanged persistentvolumeclaim/prometheus-data-pvc unchanged deployment.apps/prometheus unchanged service/prometheus-service unchanged 注意：这里prometheus是用的pv,pvc做本地存储。所以，需要在这之前需要部署好nfs，此处省略部署nfs
[root@master k8s-promethus]# kubectl get pv -n ns-monitor NAME CAPACITY ACCESSMODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE prometheus-data-pv 5Gi RWO Recycle Bound ns-monitor/prometheus-data-pvc 157m [root@master k8s-promethus]# kubectl get pvc -n ns-monitor NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE prometheus-data-pvc Bound prometheus-data-pv 5Gi RWO 158m 说明：此时pvc已经成功绑定pv. [root@master k8s-promethus]# kubectl get pod -n ns-monitor NAME READY STATUS RESTARTS AGE node-exporter-cl6d5 1/1 Running 15 3h27m node-exporter-gwwnj 1/1 Running 0 3h27m node-exporter-hbglm 1/1 Running 0 3h27m node-exporter-kwsfv 1/1 Running 0 3h27m prometheus-dd69c4889-qwmww 1/1 Running 0 161m [root@master k8s-promethus]# kubectl get svc -n ns-monitor NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE node-exporter-service NodePort 10.99.128.173 <none> 9100:30372/TCP 3h29m prometheus-service NodePort 10.96.119.235 <none> 9090:31555/TCP 162m 访问测试：
说明：prometheus已经成功运行。
4.部署kube-state-metrics: wget https://github.com/kubernetes/kube-state-metrics/tree/master/examples/standard [root@master kube-state-metrics]# for file in cluster-role-binding.yaml cluster-role.yaml service-account.yaml service.yaml deployment.yaml ; do wget https://raw.githubusercontent.com/kubernetes/kube-state-metrics/master/examples/standard/$file ;done [root@master kube-state-metrics]# ll 总用量 24 -rw-r--r-- 1 root root 376 12月 8 20:29 cluster-role-binding.yaml -rw-r--r-- 1 root root 1651 12月 8 20:14 cluster-role.yaml -rw-r--r-- 1 root root 1068 12月 8 20:29 deployment.yaml -rw-r--r-- 1 root root 634 12月 8 21:19 kube-state-svc.yaml -rw-r--r-- 1 root root 192 12月 8 20:29 service-account.yaml -rw-r--r-- 1 root root 452 12月 8 21:36 service.yaml [root@master kube-state-metrics]# for i in cluster-role-binding.yaml deployment.yaml service-account.yaml service.yaml; do sed -i '/namespace/s/kube-system/ns-monitor/' $i ; done 注意：修改下来的需要修改与自己对应的名称空间 修改service.yaml [root@master kube-state-metrics]# vim service.yaml apiVersion: v1 kind: Service metadata: labels: app.kubernetes.io/name: kube-state-metrics app.kubernetes.io/version: v1.8.0 name: kube-state-metrics namespace: ns-monitor annotations: prometheus.io/scrape: "true" ##添加此参数，允许prometheus自动发现 spec: clusterIP: None ports: - name: http-metrics port: 8080 targetPort: http-metrics - name: telemetry port: 8081 targetPort: telemetry selector: app.kubernetes.io/name: kube-state-metrics [root@master kube-state-metrics]# kubectl apply -f ./ [root@master kube-state-metrics]# kubectl get pod -n ns-monitor NAME READY STATUS RESTARTS AGE grafana-576db894c6-4qtcl 1/1 Running 0 23h kube-state-metrics-7b48c458ff-vwd5p 1/1 Running 0 137m node-exporter-cl6d5 1/1 Running 15 24h node-exporter-gwwnj 1/1 Running 0 24h node-exporter-hbglm 1/1 Running 1 24h node-exporter-kwsfv 1/1 Running 0 24h prometheus-dd69c4889-qwmww 1/1 Running 0 23h [root@master kube-state-metrics]# kubectl get svc -n ns-monitor NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE grafana-service NodePort 10.104.174.223 <none> 3000:30771/TCP 23h kube-state-metrics ClusterIP None <none> 8080/TCP,8081/TCP 112m node-exporter-service NodePort 10.99.128.173 <none> 9100:30372/TCP 24h prometheus-service NodePort 10.96.119.235 <none> 9090:31555/TCP 23h 说明：此时kube-state-metrics已经运行起来。 在prometheus查看：
说明:kube-state-metrics已经成功被prometheus监控。
5.在kubernetest中部署grafana： [root@master k8s-promethus]# cat grafana.yaml apiVersion: v1 kind: PersistentVolume metadata: name: "grafana-data-pv" labels: name: grafana-data-pv release: stable spec: capacity: storage: 5Gi accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Recycle nfs: path: /data/volumes/v2 server: 192.168.100.64 --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: grafana-data-pvc namespace: ns-monitor spec: accessModes: - ReadWriteOnce resources: requests: storage: 5Gi selector: matchLabels: name: grafana-data-pv release: stable --- kind: Deployment apiVersion: apps/v1 metadata: labels: app: grafana name: grafana namespace: ns-monitor spec: replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: app: grafana template: metadata: labels: app: grafana spec: securityContext: runAsUser: 0 containers: - name: grafana image: grafana/grafana:latest imagePullPolicy: IfNotPresent env: - name: GF_AUTH_BASIC_ENABLED value: "true" - name: GF_AUTH_ANONYMOUS_ENABLED value: "false" readinessProbe: httpGet: path: /login port: 3000 volumeMounts: - mountPath: /var/lib/grafana name: grafana-data-volume ports: - containerPort: 3000 protocol: TCP volumes: - name: grafana-data-volume persistentVolumeClaim: claimName: grafana-data-pvc --- kind: Service apiVersion: v1 metadata: labels: app: grafana name: grafana-service namespace: ns-monitor spec: ports: - port: 3000 targetPort: 3000 selector: app: grafana type: NodePort [root@master k8s-promethus]# kubectl apply -f grafana.yaml persistentvolume/grafana-data-pv unchanged persistentvolumeclaim/grafana-data-pvc unchanged deployment.apps/grafana unchanged service/grafana-service unchanged 注意：此时grafana也需要pvc. [root@master k8s-promethus]# kubectl get pv,pvc -n ns-monitor NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE persistentvolume/grafana-data-pv 5Gi RWO Recycle Bound ns-monitor/grafana-data-pvc 164m persistentvolume/prometheus-data-pv 5Gi RWO Recycle Bound ns-monitor/prometheus-data-pvc 168m NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE persistentvolumeclaim/grafana-data-pvc Bound grafana-data-pv 5Gi RWO 164m persistentvolumeclaim/prometheus-data-pvc Bound prometheus-data-pv 5Gi RWO 168m [root@master k8s-promethus]# kubectl get pod -n ns-monitor NAME READY STATUS RESTARTS AGE grafana-576db894c6-4qtcl 1/1 Running 0 166m node-exporter-cl6d5 1/1 Running 15 3h36m node-exporter-gwwnj 1/1 Running 0 3h36m node-exporter-hbglm 1/1 Running 0 3h36m node-exporter-kwsfv 1/1 Running 0 3h36m prometheus-dd69c4889-qwmww 1/1 Running 0 169m [root@master k8s-promethus]# kubectl get svc -n ns-monitor NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE grafana-service NodePort 10.104.174.223 <none> 3000:30771/TCP 166m node-exporter-service NodePort 10.99.128.173 <none> 9100:30372/TCP 3h37m prometheus-service NodePort 10.96.119.235 <none> 9090:31555/TCP 170m 访问测试：
说明：Grafana已经部署完成，用户名/密码:admin/admin
添加数据源：URL=http://prometheus-service.ns-monitor.svc.cluster.local:9090
导入模板：
