Centos7二进制部署k8s-v1.20.2 ipvs版本(Prometheus监控k8s)
一、k8s部署Prometheus
获取最新更新以及文章用到的软件包，请移步点击：查看更新
1、组件说明
1）MetricServer：是kubernetes集群资源使用情况的聚合器，收集数据给kubernetes集群内使用，如kubectl,hpa,scheduler等。
2）PrometheusOperator：是一个系统监测和警报工具箱，用来存储监控数据。
3）NodeExporter：用于各node的关键度量指标状态数据。
4）KubeStateMetrics：收集kubernetes集群内资源对象数据，制定告警规则。
5）Prometheus：采用pull方式收集apiserver，scheduler，controller-manager，kubelet组件数据，通过http协议传输。
6）Grafana：是可视化数据统计和监控平台。
2、git上拉工程
git clone https://github.com/coreos/kube-prometheus.git
3、修改容器访问方式
cd /root/kube-prometheus/manifests # 修改 grafana-service.yaml 文件，使用 nodepode 方式访问 grafana apiVersion: v1 kind: Service metadata: labels: app.kubernetes.io/component: grafana app.kubernetes.io/name: grafana app.kubernetes.io/part-of: kube-prometheus app.kubernetes.io/version: 8.0.3 name: grafana namespace: monitoring spec: type: NodePort #添加内容 ports: - name: http port: 3000 targetPort: http nodePort: 30100 #添加内容 selector: app.kubernetes.io/component: grafana app.kubernetes.io/name: grafana app.kubernetes.io/part-of: kube-prometheus # 修改 prometheus-service.yaml，改为 nodepode apiVersion: v1 kind: Service metadata: labels: app.kubernetes.io/component: prometheus app.kubernetes.io/name: prometheus app.kubernetes.io/part-of: kube-prometheus app.kubernetes.io/version: 2.28.1 prometheus: k8s name: prometheus-k8s namespace: monitoring spec: type: NodePort #添加内容 ports: - name: web port: 9090 targetPort: web nodePort: 30200 #添加内容 selector: app: prometheus app.kubernetes.io/component: prometheus app.kubernetes.io/name: prometheus app.kubernetes.io/part-of: kube-prometheus prometheus: k8s sessionAffinity: ClientIP # 修改 alertmanager-service.yaml，改为 nodepode apiVersion: v1 kind: Service metadata: labels: alertmanager: main app.kubernetes.io/component: alert-router app.kubernetes.io/name: alertmanager app.kubernetes.io/part-of: kube-prometheus app.kubernetes.io/version: 0.22.2 name: alertmanager-main namespace: monitoring spec: type: NodePort #添加内容 ports: - name: web port: 9093 targetPort: web nodePort: 30300 #添加内容 selector: alertmanager: main app: alertmanager app.kubernetes.io/component: alert-router app.kubernetes.io/name: alertmanager app.kubernetes.io/part-of: kube-prometheus sessionAffinity: ClientIP
4、修改镜像地址，国外有些镜像拉不下来
# 修改kube-state-metrics-deployment.yaml中的镜像地址 apiVersion: apps/v1 kind: Deployment metadata: labels: app.kubernetes.io/component: exporter app.kubernetes.io/name: kube-state-metrics app.kubernetes.io/part-of: kube-prometheus app.kubernetes.io/version: 2.1.1 name: kube-state-metrics namespace: monitoring spec: replicas: 1 selector: matchLabels: app.kubernetes.io/component: exporter app.kubernetes.io/name: kube-state-metrics app.kubernetes.io/part-of: kube-prometheus template: metadata: annotations: kubectl.kubernetes.io/default-container: kube-state-metrics labels: app.kubernetes.io/component: exporter app.kubernetes.io/name: kube-state-metrics app.kubernetes.io/part-of: kube-prometheus app.kubernetes.io/version: 2.1.1 spec: containers: - args: - --host=127.0.0.1 - --port=8081 - --telemetry-host=127.0.0.1 - --telemetry-port=8082 image: bitnami/kube-state-metrics:latest #修改镜像地址 name: kube-state-metrics resources: limits: cpu: 100m memory: 250Mi requests: cpu: 10m memory: 190Mi securityContext: runAsUser: 65534 - args: - --logtostderr - --secure-listen-address=:8443 - --tls-cipher-suites=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305 - --upstream=http://127.0.0.1:8081/ image: quay.io/brancz/kube-rbac-proxy:v0.11.0 name: kube-rbac-proxy-main ports: - containerPort: 8443 name: https-main resources: limits: cpu: 40m memory: 40Mi requests: cpu: 20m memory: 20Mi securityContext: runAsGroup: 65532 runAsNonRoot: true runAsUser: 65532 - args: - --logtostderr - --secure-listen-address=:9443 - --tls-cipher-suites=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305 - --upstream=http://127.0.0.1:8082/ image: quay.io/brancz/kube-rbac-proxy:v0.11.0 name: kube-rbac-proxy-self ports: - containerPort: 9443 name: https-self resources: limits: cpu: 20m memory: 40Mi requests: cpu: 10m memory: 20Mi securityContext: runAsGroup: 65532 runAsNonRoot: true runAsUser: 65532 nodeSelector: kubernetes.io/os: linux serviceAccountName: kube-state-metrics # 修改prometheus-prometheus.yaml镜像地址 apiVersion: monitoring.coreos.com/v1 kind: Prometheus metadata: labels: app.kubernetes.io/component: prometheus app.kubernetes.io/name: prometheus app.kubernetes.io/part-of: kube-prometheus app.kubernetes.io/version: 2.28.1 prometheus: k8s name: k8s namespace: monitoring spec: alerting: alertmanagers: - apiVersion: v2 name: alertmanager-main namespace: monitoring port: web enableFeatures: [] externalLabels: {} image: registry.cn-hangzhou.aliyuncs.com/zenggh/prometheus:v2.28.1 #修改此处镜像 nodeSelector: kubernetes.io/os: linux podMetadata: labels: app.kubernetes.io/component: prometheus app.kubernetes.io/name: prometheus app.kubernetes.io/part-of: kube-prometheus app.kubernetes.io/version: 2.28.1 podMonitorNamespaceSelector: {} podMonitorSelector: {} probeNamespaceSelector: {} probeSelector: {} replicas: 2 resources: requests: memory: 400Mi ruleNamespaceSelector: {} ruleSelector: matchLabels: prometheus: k8s role: alert-rules securityContext: fsGroup: 2000 runAsNonRoot: true runAsUser: 1000 serviceAccountName: prometheus-k8s serviceMonitorNamespaceSelector: {} serviceMonitorSelector: {} version: 2.28.1
5、创建容器
cd /root/kube-prometheus/manifests kubectl create namespace monitoring kubectl apply -f . kubectl apply -f ./setup
6、等待容器创建成功，浏览器访问
Prometheus访问地址：http://ip:30200/
Grafana访问地址：http://ip:30100/ 默认账号密码：admin/admin
7、Grafana添加监控项
1)添加数据源
2）选择源数据
3）测试连通性
4）导入监控项
5）查看
6）选择查看监控项
7）查看集群节点资源占用率
