linux运维、架构之路-K8s数据管理
一、Volume介绍
容器和Pod是短暂的，它们的生命周期可能很短，会被频繁的销毁和创建，存在容器中的数据会被清除，为了持久化保存容器的数据，k8s提供了Volume。
Volume的生命周期独立于容器，它是一个目录，Volume会被mount到Pod，Pod中的所有容器都可以访问这个Volume，和Docker Volume类似。
Volume支持的类型：emptyDir、hostPath、AWS Elastic Block Store、NFS、Ceph等。
1、emptyDir
emptyDir是最基础的Volume类型，它对于容器来说是持久的，对于Pod刚不是，当Pod从节点删除时Volume中的数据也会被删除，如果只是容器被销毁而Pod还在，则Volume不受影响。
①创建示例应用
apiVersion: v1 kind: Pod metadata: name: hello-world spec: containers: - image: busybox name: hello volumeMounts: - mountPath: /hello_dir #②hello容器将shared-volume mount到/hello_dir目录 name: shared-volume args: - /bin/sh #③hello通过echo将数据写到文件hello里 - -c - echo "hello world" >/hello_dir/hello; sleep 30000 - image: busybox name: world #④world容器将 shared-volume mount到 /world_dir目录 volumeMounts: - mountPath: /world_dir name: shared-volume args: - /bin/sh #⑤world通过cat从文件hello读取数据 - -c - cat /world_dir/hello; sleep 30000 volumes: - name: shared-volume #①定义了一个emptyDir类型的Volume shared-volume emptyDir: {}
②查看验证
[root@k8s-node1 volume]# kubectl apply -f emptyDir.yaml pod "hello-world" created [root@k8s-node1 volume]# kubectl get pod NAME READY STATUS RESTARTS AGE hello-world 2/2 Running 0 36s [root@k8s-node1 volume]# kubectl logs hello-world world hello world
上述显示容器world成功读到了hello容器写入的数据，验证了两个容器共享emptyDir Volume,其效果相当于执行了docker run -v /hello_dir 和/docker run -v /world_dir
emptyDir是Host主机上创建的临时目录，其优点能够方便地为Pod中的容器提供共享存储，但不具有持久性，如果Pod不存在了，emptyDir也就没有了，适合Pod中容器需要临时共享存储空间的应用场景
2、HostPath
hostPath Volume的作用是将Docker Host文件系统中已经存在的目录mount给Pod的容器，大部分应用不会使用此类型，因为这实际上增加上了Pod与节点的耦合，有些需要访问k8s内部数据，比如配置文件和二进制库这样的应用需要使用hostPath，例如k8s自身的应用kube-apiserver和kube-controller-manager。
3、外部Storage Provider
如果k8s部署在AWS、GCE、Azure等公有云上，可以直接使用云盘作为Volume。因资源有限，此处无法演示。
二、PV & PVC
- PV是外部存储系统中的一块存储空间，由管理员创建，与Volume一样具有持久性，生命周期独立于Pod。
- PVC刚是对PV的申请，PVC通常由普通用户创建，需要为Pod分配存储资源时，用户可以创建一个PVC，指明存储资源的容量大小和访问权限等，k8s会查找并提供满足条件的PV。
- k8s支持多种类型的PV，比如AWS EBS、Ceph、NFS等
1、NFS PV
①在k8s-master节点搭建一个NFS服务，目录为/nfsdata
[root@k8s-node1 ~]# showmount -e Export list for k8s-node1: /nfsdata *
NFS服务具体部署请参照：https://www.cnblogs.com/yanxinjiang/p/7419469.html
②创建应用示例
apiVersion: v1 kind: PersistentVolume metadata: name: mypv spec: capacity: storage: 1Gi #指定PV的容量为1GB accessModes: - ReadWriteOnce #表示PV能为read-write模式mount到单个节点，ReadOnlyMany表示只读，mount到多个节点，ReadWriteMany表示读写可以mount到多节点 persistentVolumeReclaimPolicy: Recycle #PV的回收策略，清除PV中的数据，Retain表示管理员手工回收 storageClassName: nfs #指定PV的class为nfs，相当于为PV设置了一个分类，PVC可以指定class申请相应的PV nfs: path: /nfsdata/pv1 #指定PV在NFS服务器上对应的目录 server: 192.168.56.11
[root@k8s-node1 nfs-pv]# kubectl apply -f nfs-pv1.yaml persistentvolume "mypv1" created [root@k8s-node1 nfs-pv]# kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE #STATUS为Available，表示mypv1就绪，可以被PVC申请 mypv1 1Gi RWO Recycle Available nfs 4s
③创建PVC
kind: PersistentVolumeClaim apiVersion: v1 metadata: name: mypv spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi storageClassName: nfs
[root@k8s-node1 nfs-pv]# kubectl apply -f nfs-pvc1.yaml persistentvolumeclaim "mypvc1" created [root@k8s-node1 nfs-pv]# kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE mypvc1 Bound mypv1 1Gi RWO nfs 13s [root@k8s-node1 nfs-pv]# kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE mypv1 1Gi RWO Recycle Bound default/mypvc1 nfs 22m
结果：mypvc1已经成功绑定到mypv1上面
④在Pod中使用存储
kind: Pod apiVersion: v1 metadata: name: mypod1 spec: containers: - name: mypod1 image: busybox args: - /bin/sh - -c - sleep 30000 volumeMounts: - mountPath: "/mydata" name: mypv volumes: - name: mypv persistentVolumeClaim: claimName: mypv
⑤测试验证
[root@k8s-node1 nfs-pv]# kubectl apply -f pod1.yaml [root@k8s-node1 nfs-pv]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE mypod1 1/1 Running 0 11m 10.2.83.14 192.168.56.12 [root@k8s-node1 nfs-pv]# kubectl exec mypod1 touch /mydata/readme.md [root@k8s-node1 nfs-pv]# ls /nfsdata/pv1/ readme.md
在Pod中创建的文件/mydata已经存储到了NFS服务器目录/nfsdata/pv1中。
⑥回收PV
- 当不需要使用PV时，可用删除PVC回收PV
[root@k8s-node1 nfs-pv]# kubectl delete pvc mypvc1 persistentvolumeclaim "mypvc1" deleted
因PV的回收策略设置为Recycle，所以数据会被清除，如想保存数据，可以将策略设置为Retain
apiVersion: v1 kind: PersistentVolume metadata: name: mypv1 spec: capacity: storage: 1Gi accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Retain #Recycle storageClassName: nfs nfs: path: /nfsdata/pv1 server: 192.168.56.11
[root@k8s-node1 nfs-pv]# kubectl apply -f nfs-pv1.yaml persistentvolume "mypv1" configured [root@k8s-node1 nfs-pv]# kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE mypv1 1Gi RWO Retain Bound default/mypvc1 nfs
⑦验证
[root@k8s-node1 nfs-pv]# kubectl apply -f nfs-pvc1.yaml [root@k8s-node1 nfs-pv]# kubectl exec mypod1 touch /mydata/hello.world [root@k8s-node1 nfs-pv]# ls /nfsdata/pv1/ hello.world [root@k8s-node1 nfs-pv]# kubectl delete pvc mypvc1 persistentvolumeclaim "mypvc1" deleted [root@k8s-node1 nfs-pv]# ls /nfsdata/pv1/ hello.world
2、PV动态供给
- 静态供给：我们提前创建PV，通过PVC申请PV并在Pod中使用
- 动态供给：如果没有满足PVC条件的PV，会动态创建PV，不需要提前创建PV，效率高，通过StorageClass实现。
3、k8s中MySQL数据持久化存储
①创建PV和PVC
apiVersion: v1 kind: PersistentVolume metadata: name: mysql-pv spec: capacity: storage: 2Gi accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Retain storageClassName: nfs nfs: server: 192.168.56.11 path: /data/k8s/mysql-pv --- kind: PersistentVolumeClaim apiVersion: v1 metadata: name: mysql-pvc spec: accessModes: - ReadWriteOnce resources: requests: storage: 2Gi storageClassName: nfs
查看pv,pvc状态
[root@k8s-node1 mysql]# kubectl apply -f mysql_pv.yaml persistentvolume "mysql-pv" created persistentvolumeclaim "mysql-pvc" created [root@k8s-node1 mysql]# kubectl get pv,pvc NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE persistentvolume/mysql-pv 2Gi RWO Retain Bound default/mysql-pvc nfs 4s NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE persistentvolumeclaim/mysql-pvc Bound mysql-pv 2Gi RWO nfs 4s
②部署MySQL
apiVersion: apps/v1beta1 kind: Deployment metadata: name: mysql spec: selector: matchLabels: app: mysql template: metadata: labels: app: mysql spec: containers: - name: mysql image: mysql:5.6 imagePullPolicy: IfNotPresent env: - name: MYSQL_ROOT_PASSWORD value: password ports: - name: mysql containerPort: 3306 volumeMounts: - mountPath: /var/lib/mysql name: mysql-persistent-storage volumes: - name: mysql-persistent-storage persistentVolumeClaim: claimName: mysql-pvc --- apiVersion: v1 kind: Service metadata: name: mysql spec: ports: - port: 3306 selector: app: mysql
创建应用查看
[root@k8s-node1 mysql]# kubectl apply -f mysql.yaml deployment.apps "mysql" created [root@k8s-node1 k8s]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE mysql-57c44d9d5f-5v7qc 1/1 Running 0 7m 10.2.8.25 192.168.56.13
③进入MySQL数据库创建测试数据
kubectl exec -it mysql-57c44d9d5f-5v7qc /bin/bash root@mysql-57c44d9d5f-5v7qc:/# mysql -uroot -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.6.46 MySQL Community Server (GPL) Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> mysql> mysql> mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema |
mysql> create database test; Query OK, 1 row affected (0.01 sec) mysql> use test; Database changed mysql> create table my_id(id int(4)); Query OK, 0 rows affected (0.03 sec) mysql> insert my_id values(111); Query OK, 1 row affected (0.01 sec) mysql> select * from my_id; +------+ | id | +------+ | 111 | +------+ 1 row in set (0.00 sec)
④模拟k8s-node3节点故障
此时node3节点变为不可用
[root@k8s-node1 mysql]# kubectl get node NAME STATUS ROLES AGE VERSION 192.168.56.12 Ready <none> 6d v1.10.3 192.168.56.13 NotReady <none> 6d v1.10.3
⑤等待一段时间后，k8s会将MySQL迁移至node2节点
[root@k8s-node1 mysql]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE mysql-57c44d9d5f-5v7qc 1/1 Unknown 0 47m 10.2.8.25 192.168.56.13 mysql-57c44d9d5f-6gvh5 1/1 Running 0 18m 10.2.93.74 192.168.56.12 [root@k8s-node1 mysql]# kubectl exec -it mysql-57c44d9d5f-6gvh5 /bin/bash
⑥再次登录MySQL验证数据的一致性
root@mysql-57c44d9d5f-6gvh5:/# mysql -uroot -ppassword Warning: Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.6.46 MySQL Community Server (GPL) Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | test | +--------------------+ 4 rows in set (0.02 sec) mysql> use test; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A mysql> show tables; +----------------+ | Tables_in_test | +----------------+ | my_id | +----------------+ 1 row in set (0.00 sec) mysql> select * from my_id; +------+ | id | +------+ | 111 | +------+ 1 row in set (0.00 sec)
结果：MySQL服务恢复，数据无丢失
成功最有效的方法就是向有经验的人学习！
