OpenEBS 实现 Local PV 动态持久化存储
OpenEBS(https://openebs.io) 是一种模拟了 AWS 的 EBS、阿里云的云盘等块存储实现的基于容器的存储开源软件。OpenEBS 是一种基于 CAS(Container Attached Storage) 理念的容器解决方案，其核心理念是存储和应用一样采用微服务架构，并通过 Kubernetes 来做资源编排。其架构实现上，每个卷的 Controller 都是一个单独的 Pod，且与应用 Pod 在同一个节点，卷的数据使用多个 Pod 进行管理。
OpenEBS 有很多组件，可以分为以下几类：
1.控制平面组件 - 管理 OpenEBS 卷容器，通常会用到容器编排软件的功能
2.数据平面组件 - 为应用程序提供数据存储，包含 Jiva 和 cStor 两个存储后端
3.节点磁盘管理器 - 发现、监控和管理连接到 Kubernetes 节点的媒体
4. 与云原生工具的整合 - 与 Prometheus、Grafana、Fluentd 和 Jaeger 进行整合。
选择合适的存储引擎
可能自己到官网看文档了比我讲的清楚
https://docs.openebs.io/#replicated-volumes-aka-highly-available-volumes
上机实践
所有节点安装启动 iSCSI 启动器
yum install iscsi-initiator-utils -y
systemctl enable --now iscsid
systemctl start iscsid.service
systemctl status iscsid.service
安装 OpenEBS
kubectl apply -f https://openebs.github.io/charts/openebs-operator.yaml
kubectl get pods -n openebs
默认情况下 OpenEBS 还会安装一些内置的 StorageClass 对象：
kubectl get sc
直接使用上面自带的 openebs-hostpath 这个 StorageClass 来创建 PVC：
cat<<END> local-hostpath-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: local-hostpath-pvc
namespace: demon
spec:
storageClassName: openebs-hostpath
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 3Gi
END
kubectl apply -f local-hostpath-pvc.yaml
PVC 的状态是 Pending，这是因为对应的 StorageClass 是延迟绑定模式，所以需要等到 Pod 消费这个 PVC 后才会去绑定
资源清单
cat<<END>nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-gb
namespace: demon
#namespace: cephfs
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:alpine
ports:
- containerPort: 80
volumeMounts:
- mountPath: /usr/share/nginx/html
name: data
volumes:
- name: data
persistentVolumeClaim:
claimName: local-hostpath-pvc
END
创建一个文件测试
echo 555555555 > /usr/share/nginx/html/index.html
1
tee redis.yaml<<-'EOF'
apiVersion: v1
kind: Service
metadata:
name: redis
namespace: test01
labels:
app: redis
spec:
ports:
- port: 6379
targetPort: 6379
name: redis-port
clusterIP: None
selector:
app: redis
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: redis
namespace: test01
spec:
selector:
matchLabels:
app: redis
serviceName: "redis-service"
replicas: 1
template:
metadata:
labels:
app: redis
spec:
terminationGracePeriodSeconds: 20
containers:
- name: redis
image: redis:6.2.6-alpine
command:
- "redis-server"
args:
- "/etc/redis/redis.conf" #redis-server后面跟的参数,换行代表空格
- "--protected-mode" #允许外网访问
- "no"
# command: redis-server /etc/redis/redis.conf --protected-mode no
resources:
requests:
cpu: "100m"
memory: "100Mi"
limits:
cpu: 4000m
memory: 4Gi
ports:
- name: redis
containerPort: 6379
protocol: "TCP"
volumeMounts:
- name: "redis-conf"
mountPath: "/etc/redis"
- name: "redis-data"
mountPath: "/var/lib/redis"
volumes:
- name: "redis-conf"
configMap:
name: "redis-conf"
items:
- key: "redis.conf"
path: "redis.conf"
volumeClaimTemplates:
- metadata:
name: "redis-data"
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: openebs-hostpath
resources:
requests:
storage: 1Gi
---
apiVersion: v1
kind: ConfigMap
metadata:
name: redis-conf
namespace: test01
data:
redis.conf: |
bind 0.0.0.0
port 6379
requirepass 111111
dbfilename dump_6379.rdb
dir /var/lib/redis
logfile /tmp/redis-6379.log
protected-mode no
EOF
#kubectl -n demon get pv pvc-fd9ffa47-6d9d-4adc-97d6-75de54951717 -o yaml
apiVersion: v1
kind: PersistentVolume
metadata:
annotations:
pv.kubernetes.io/provisioned-by: openebs.io/local
creationTimestamp: "2021-01-08T03:16:17Z"
finalizers:
- kubernetes.io/pv-protection
labels:
openebs.io/cas-type: local-hostpath
managedFields:
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:status:
f:phase: {}
manager: kube-controller-manager
operation: Update
time: "2021-01-08T03:16:17Z"
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.: {}
f:pv.kubernetes.io/provisioned-by: {}
f:labels:
.: {}
f:openebs.io/cas-type: {}
f:spec:
f:accessModes: {}
f:capacity:
.: {}
f:storage: {}
f:claimRef:
.: {}
f:apiVersion: {}
f:kind: {}
f:name: {}
f:namespace: {}
f:resourceVersion: {}
f:uid: {}
f:local:
.: {}
f:fsType: {}
f:path: {}
f:nodeAffinity:
.: {}
f:required:
.: {}
f:nodeSelectorTerms: {}
f:persistentVolumeReclaimPolicy: {}
f:storageClassName: {}
f:volumeMode: {}
manager: provisioner-localpv
operation: Update
time: "2021-01-08T03:16:17Z"
name: pvc-fd9ffa47-6d9d-4adc-97d6-75de54951717
resourceVersion: "1359440"
uid: bd30ddba-03a6-4230-b493-101e12ad5787
spec:
accessModes:
- ReadWriteOnce
capacity:
storage: 3Gi
claimRef:
apiVersion: v1
kind: PersistentVolumeClaim
name: local-hostpath-pvc
namespace: demon
resourceVersion: "1359413"
uid: fd9ffa47-6d9d-4adc-97d6-75de54951717
local:
fsType: ""
path: /var/openebs/local/pvc-fd9ffa47-6d9d-4adc-97d6-75de54951717
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- node63
persistentVolumeReclaimPolicy: Delete
storageClassName: openebs-hostpath
volumeMode: Filesystem
status:
phase: Bound
可以看到这个自动生成的 PV 和我们前面自己手动创建的 Local PV 基本上是一致的，和 node2 节点是亲和关系，本地数据目录位于 /var/openebs/local/pvc-fd9ffa47-6d9d-4adc-97d6-75de54951717
验证下 volume 数据，前往node63 节点查看下上面的数据目录中的数据：
看到 Pod 容器中的数据已经持久化到 Local PV 对应的目录中去了。但是需要注意的是 StorageClass 默认的数据回收策略是 Delete，所以如果将 PVC 删掉后数据会自动删除，我们可以 Velero 这样的工具来进行备份还原
————————————————
版权声明：本文为CSDN博主「野猪佩挤」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。
原文链接：https://blog.csdn.net/weixin_42562106/article/details/112347574
