k8s-Longhorn系统配置 20241017 -分布式存储
一 Longhorn存储部署
1.1 Longhorn概述
Longhorn是用于Kubernetes的开源分布式块存储系统。
提示：更多介绍参考：https://github.com/longhorn/longhorn。
1.2 Longhorn部署
1 [root@master01 ~]# cd /opt/k8s/work/
2 [root@master01 work]# source /root/environment.sh
3 [root@master01 work]# for all_ip in ${ALL_IPS[@]}
4 do
5 echo ">>> ${all_ip}"
6 ssh root@${all_ip} "yum -y install iscsi-initiator-utils &"
7 done
提示：本步骤操作仅需要在master01节点操作。
1 [root@master01 work]# mkdir longhorn
2 [root@master01 work]# cd longhorn/
3 [root@master01 longhorn]# wget \
4 https://raw.githubusercontent.com/longhorn/longhorn/master/deploy/longhorn.yaml
5 [root@master01 longhorn]# vi longhorn.yaml
1 #……
2 ---
3 kind: Service
4 apiVersion: v1
5 metadata:
6 labels:
7 app: longhorn-ui
8 name: longhorn-frontend
9 namespace: longhorn-system
10 spec:
11 type: NodePort #修改为nodeport
12 selector:
13 app: longhorn-ui
14 ports:
15 - port: 80
16 targetPort: 8000
17 nodePort: 30002
18 ---
19 ……
20 kind: DaemonSet
21 ……
22 imagePullPolicy: IfNotPresent
23 ……
24 #……
1 [root@master01 longhorn]# kubectl apply -f longhorn.yaml
2 [root@master01 longhorn]# kubectl -n longhorn-system get pods -o wide
提示：若部署异常可删除重建，若出现无法删除namespace，可通过如下操作进行删除：
1 wget https://github.com/longhorn/longhorn/blob/master/uninstall/uninstall.yaml
2 rm -rf /var/lib/longhorn/
3 kubectl apply -f uninstall.yaml
4 kubectl delete -f longhorn.yaml
1.5 动态sc创建
提示：默认longhorn部署完成已创建一个sc，也可通过如下手动编写yaml创建。
1 [root@master01 longhorn]# kubectl get sc
2 NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
3 ……
4 longhorn driver.longhorn.io Delete Immediate true 15m
5 [root@master01 longhorn]# vi longhornsc.yaml
1 kind: StorageClass
2 apiVersion: storage.k8s.io/v1
3 metadata:
4 name: longhornsc
5 provisioner: rancher.io/longhorn
6 parameters:
7 numberOfReplicas: "3"
8 staleReplicaTimeout: "30"
9 fromBackup: ""
1 [root@master01 longhorn]# kubectl create -f longhornsc.yaml
1.6 测试PV及PVC
1 [root@master01 longhorn]# vi longhornpod.yaml
1 apiVersion: v1
2 kind: PersistentVolumeClaim
3 metadata:
4 name: longhorn-pvc
5 spec:
6 accessModes:
7 - ReadWriteOnce
8 storageClassName: longhorn
9 resources:
10 requests:
11 storage: 2Gi
12 ---
13 apiVersion: v1
14 kind: Pod
15 metadata:
16 name: longhorn-pod
17 namespace: default
18 spec:
19 containers:
20 - name: volume-test
21 image: nginx:stable-alpine
22 imagePullPolicy: IfNotPresent
23 volumeMounts:
24 - name: volv
25 mountPath: /data
26 ports:
27 - containerPort: 80
28 volumes:
29 - name: volv
30 persistentVolumeClaim:
31 claimName: longhorn-pvc
1 [root@master01 longhorn]# kubectl apply -f longhornpod.yaml
2 [root@master01 longhorn]# kubectl get pods -o wide
3 [root@master01 longhorn]# kubectl get pvc
4 [root@master01 longhorn]# kubectl get pv
提示：本步骤操作仅需要在master01节点操作。
1.7 Ingress暴露Longhorn
1 [root@master01 longhorn]# yum -y install httpd-tools
2 [root@master01 longhorn]# htpasswd -c auth xhy #创建用户名和密码
提示：也可通过如下命令创建：
1 [root@master01 longhorn]# kubectl -n longhorn-system create secret generic longhorn-basic-auth --from-file=auth
2
3 [root@master01 longhorn]# vi longhorn-ingress.yaml #创建ingress规则
1 apiVersion: networking.k8s.io/v1beta1
2 kind: Ingress
3 metadata:
4 name: longhorn-ingress
5 namespace: longhorn-system
6 annotations:
7 kubernetes.io/ingress.class: "nginx"
8 nginx.ingress.kubernetes.io/auth-type: basic
9 nginx.ingress.kubernetes.io/auth-secret: longhorn-basic-auth
10 nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required '
11 spec:
12 rules:
13 - host: longhorn.odocker.com
14 http:
15 paths:
16 - path: /
17 backend:
18 serviceName: longhorn-frontend
19 servicePort: 80
1 [root@master01 longhorn]# kubectl apply -f longhorn-ingress.yaml
提示：本步骤操作仅需要在master01节点操作。
1.8 确认验证
浏览器访问：longhorn.odocker.com，并输入账号和密码。
登录查看。
附加 Helm部署
附0.1 helm安装
Helm 是 Kubernetes 的软件包管理工具，此处作为建议项部署，更多helm参考《053.集群管理-Helm工具》。
1 [root@master01 ~]# cd /opt/k8s/work/
2 [root@master01 work]# mkdir /opt/k8s/work/helm
3 [root@master01 work]# source /root/environment.sh
4 [root@master01 work]# wget http://down.linuxsb.com:8888/helm-v3.2.3-linux-amd64.tar.gz
5 [root@master01 work]# tar -zxvf helm-v3.2.3-linux-amd64.tar.gz -C /opt/k8s/work/helm
6 [root@master01 work]# for master_ip in ${MASTER_IPS[@]}
7 do
8 echo ">>> ${master_ip}"
9 scp -rp /opt/k8s/work/helm/linux-amd64/helm root@${master_ip}:/opt/k8s/bin/
10 ssh root@${master_ip} "chmod +x /opt/k8s/bin/*"
11 ssh root@${master_ip} "helm version"
12 ssh root@${master_ip} "echo 'source <(helm completion bash)' >> $HOME/.bashrc"
13 done
附0.2 helm安装
1 [root@master01 work]# helm repo add brigade https://brigadecore.github.io/charts
2 [root@master01 work]# helm repo add stable https://kubernetes-charts.storage.googleapis.com/ #添加官方repo
3 [root@master01 work]# helm repo add bitnami https://charts.bitnami.com/bitnami
4 [root@master01 work]# helm repo list #查看repo
提示：本步骤操作仅需要在master01节点操作。
转https://www.cnblogs.com/itzgr/p/13207220.html
