编译安装Kubernetes 1.29 高可用集群(7)--Metrics Server节点监控和HPA配置

1.部署Metrics Server监控

1.1 在任意k8s-master节点上下载Metrics Server的创建文件

https://github.com/kubernetes-sigs/metrics-server/releases

wget https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.7.1/high-availability-1.21+.yaml

1.2 修改单机版配置文件

sed -i "s#10250#10350#g" high-availability-1.21+.yaml
sed -i "s#registry.k8s.io/metrics-server/#registry.aliyuncs.com/google_containers/#g" high-availability-1.21+.yaml

# vi metrics-server-components.yaml
### 140行
            namespaces:
            - kube-system
            topologyKey: kubernetes.io/hostname
	  hostNetwork: true    //添加此行
      containers:
      - args:
    
### 145行
args:
        - --cert-dir=/tmp
        - --secure-port=10350
        - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
        - --kubelet-use-node-status-port
        - --metric-resolution=15s  //在此行下添加以下5行内容
        - --kubelet-insecure-tls
        - --requestheader-client-ca-file=/etc/kubernetes/pki/ca.pem
        - --requestheader-username-headers=X-Remote-User
        - --requestheader-group-headers=X-Remote-Group
        - --requestheader-extra-headers-prefix=X-Remote-Extra-

### 192行
        volumeMounts:
        - mountPath: /tmp
          name: tmp-dir  在此行下添加以下2行内容
        - name: ca-ssl
          mountPath: /etc/kubernetes/pki

### 201行
      volumes:
      - emptyDir: {}
        name: tmp-dir  在此行下添加以下3行内容
      - name: ca-ssl
        hostPath:
          path: /etc/kubernetes/pki

1.3 在任意k8s-master节点应用Metrics Server的components.yaml文件

# kubectl apply -f high-availability-1.21+.yaml
serviceaccount/metrics-server created
clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created
clusterrole.rbac.authorization.k8s.io/system:metrics-server created
rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created
clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created
clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created
service/metrics-server created
deployment.apps/metrics-server created
apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created

注：如需编辑创建后的metrics-server内可使用命令
kubectl edit deploy metrics-server -n kube-system

# kubectl get pods -n kube-system -o wide | grep metrics
metrics-server-85cd857b54-5r778   1/1     Running   0              5m35s   192.168.83.221   k8s-node02   <none>           <none>
metrics-server-85cd857b54-tslbk   1/1     Running   0              5m35s   192.168.83.220   k8s-node01   <none>           <none>

# kubectl get apiservices.apiregistration.k8s.io | grep metrics
v1beta1.metrics.k8s.io                  kube-system/metrics-server   True        19m

1.4 查看监控状态

# kubectl top node
NAME         CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%   
k8s-node01   31m          3%     800Mi           60%       
k8s-node02   58m          5%     864Mi           65% 

# kubectl top pod -A
NAMESPACE           NAME                              CPU(cores)   MEMORY(bytes)   
cilium-monitoring   grafana-74b486577f-7w978          3m           108Mi           
cilium-monitoring   prometheus-58668c58df-jt9sn       2m           113Mi           
kube-system         cilium-87hd8                      5m           181Mi           
kube-system         cilium-9fdbh                      5m           176Mi           
kube-system         cilium-operator-f45f4975f-f9q7p   2m           66Mi            
kube-system         cilium-operator-f45f4975f-gw5z6   1m           58Mi            
kube-system         coredns-78d4595769-gl8nx          1m           38Mi            
kube-system         hubble-relay-84849f9dd5-59zhs     1m           23Mi            
kube-system         hubble-ui-79b7f9f4b-ccdrh         1m           29Mi            
kube-system         metrics-server-85cd857b54-5r778   3m           40Mi            
kube-system         metrics-server-85cd857b54-tslbk   2m           19Mi

2. 集群验证

2.1 创建busybox测试CoreDNS

# mkdir /opt/pods_yaml
# cat > /opt/pods_yaml/busybox.yaml <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: busybox
  namespace: default
spec:
  containers:
  - image: busybox:1.28.3
    command:
      - sleep
      - "3600"
    imagePullPolicy: IfNotPresent
    name: busybox
  restartPolicy: Always
EOF
注：因其他版本busybox有bug，所以指定使用1.28.3版本

# kubectl create -f /opt/pods_yaml/busybox.yaml
pod/busybox created

# kubectl get pod -o wide
NAME      READY   STATUS    RESTARTS   AGE   IP             NODE         NOMINATED NODE   READINESS GATES
busybox   1/1     Running   0          11m   172.31.1.171   k8s-node02   <none>           <none>

2.2 CoreDNS解析测试

# kubectl  get svc -A | grep dns
kube-system            kube-dns                               ClusterIP   10.66.0.2       <none>        53/UDP,53/TCP,9153/TCP          140d

# kubectl exec  busybox -n default -- nslookup kube-dns.kube-system
Server:    10.66.0.2
Address 1: 10.66.0.2 kube-dns.kube-system.svc.cluster.local

Name:      kube-dns.kube-system
Address 1: 10.66.0.2 kube-dns.kube-system.svc.cluster.local

# kubectl exec -i --tty busybox /bin/sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/ # nslookup www.baidu.com
Server:    10.66.0.2
Address 1: 10.66.0.2 kube-dns.kube-system.svc.cluster.local

Name:      www.baidu.com
Address 1: 2408:871a:2100:2:0:ff:b09f:237
Address 2: 2408:871a:2100:3:0:ff:b025:348d
Address 3: 110.242.68.4
Address 4: 110.242.68.3

3. HPA资源实现pod水平伸缩（自动扩缩容）

3.1 编写deployment资源清单

# cat > centos-vm.yaml << EOF 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: centos-vm
spec:
  replicas: 1
  selector:
    matchLabels:
      run: centos-vm
  template:
    metadata:
      labels:
        run: centos-vm
    spec:
      containers:
      - name: centos-vm
        image: centos:centos7.9.2009
        command:
        - tail
        - -f
        - /etc/hosts
        resources:
          requests:
            cpu: "50m"
          limits:
            cpu: "150m"
EOF

# kubectl apply -f centos-vm.yaml
# kubectl get pod | grep centos
centos-vm-599c54c7cc-8dtp4            1/1     Running   1 (11m ago)    8h

3.2 编写HPA资源清单绑定Deployment

# cat > centos-vm-hpa.yaml << EOF
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: hpa-centos-vm
spec:
  maxReplicas: 2   //扩容上线
  minReplicas: 1   //缩容下限
  scaleTargetRef:  //扩缩容的目标
    apiVersion: apps/v1
    kind: Deployment
    name: centos-vm
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 80  //cpu阈值达到80%开始扩缩容
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageValue: 150Mi  //memory阈值达到150Mi开始扩缩容
EOF

# kubectl apply -f centos-vm-hpa.yaml
# kubectl get hpa
NAME            REFERENCE              TARGETS                MINPODS   MAXPODS   REPLICAS   AGE
hpa-centos-vm   Deployment/centos-vm   2%/80%, 491520/150Mi   1         2         1          7h45m

注释：

minReplicas：最小pod实例数
maxReplicas：最大pod实例数
metrics：用于计算所需的Pod副本数量的指标列表
resource：核心指标，包含cpu和内存两种（被弹性伸缩的pod对象中容器的requests和limits中定义的指标） 

3.3 压测测试

### 进入POD
kubectl exec centos-vm-599c54c7cc-8dtp4 -it -- sh

### 安装aliyun源和epel源
yum -y install wget
wget -O /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
wget -O /etc/yum.repos.d/epel.repo https://mirrors.aliyun.com/repo/epel-7.repo

### 安装压测工具
yum -y install stress

### 开始使用命令压测pod
stress --cpu 8 --io 4 --vm 2 --vm-bytes 256M --timeout 20m

### 查看HPA资源的负载情况
kubectl get hpa -o wide

 

posted @ 2024-07-02 14:40  不倒翁Jason  阅读(78)  评论(0)  编辑  收藏  举报