持久存储卷
使用网络存储卷实现持久化存储，必须清晰了解所用到的网络存储系统的访问细节才能完成存储卷的相关的配置任务，如：NFS存储卷的server和path字段的配置就依赖于服务器地址和共享目录路径。这与kubernetes的向用户和开发隐藏底层架构的目标有所背离，对存储资源的使用最好也能像使用计算资源一样，用户和开发人员无须了解pod资源究竟运行于哪个节点，也无须了解存储系统是什么设备以及位于何处。为此，kubernetes的Persistent Volume 子系统在用户和管理员之间添加了一个抽象层，从而使得存储系统的使用和管理职能互相解耦。
参考官网：https://kubernetes.io/zh-cn/docs/concepts/storage/persistent-volumes/
一、概念
1. PV概念
PersistentVolume（PV）是指由集群管理员配置提供的某存储系统上的一段存储空间，它是对底层共享存储的抽象，将共享存储作为一种可由用户申请使用的资源，实现了“存储消费”机制。通过存储插件机制，pv支持使用多种网络存储系统或云端存储等多种后端存储系统，如Ceph、GlusterFS、NFS 等。PV是集群级别的资源，不属于任何名称空间。
2. PVC概念
PersistentVolumeClaim（PVC）持久化卷声明：是用户存储的一种声明，PVC 和 Pod 比较类似，Pod 消耗的是节点，PVC 消耗的是 PV 资源，Pod 可以请求 CPU 和内存，而 PVC 可以请求特定的存储空间和访问模式。对于真正使用存储的用户不需要关心底层的存储实现细节，只需要直接使用 PVC 即可。
3. pod存储卷、PVC、PV的调用关系
用户对PV资源的使用需要通过PersistentVolumeClaim（PVC）提出的使用申请（或称为声明）来完成绑定，是PV资源的消费者，它向PV申请特定大小的空间及访问模式（如rw或ro）,从而创建出PVC存储卷，而后再由pod资源通过PersistentVolumeClaim存储卷关联使用。
4. StorageClass概念
通过 PVC 请求到一定的存储空间也很有可能不足以满足应用对于存储设备的各种需求，而且不同的应用程序对于存储性能的要求可能也不尽相同，比如读写速度、并发性能等，为了解决这一问题，Kubernetes 又引入了一个新的资源对象：StorageClass，通过 StorageClass 的定义，管理员可以将存储资源定义为某种类型的资源，比如快速存储、慢速存储等，用户根据 StorageClass 的描述就可以非常直观的知道各种存储资源的具体特性了，通过PVC直接向意向的类别发出申请，匹配由管理员事先创建的PV，或者由其按需为用户动态创建PV，这样做甚至免去了需要事先创建PV的过程。
二、创建PV
1. PV的资源清单字段说明
查看定义pv需要的字段
[root@k8s-master1 ~]# kubectl explain pv
KIND: PersistentVolume
VERSION: v1
DESCRIPTION:
PersistentVolume (PV) is a storage resource provisioned by an
administrator. It is analogous to a node. More info:
https://kubernetes.io/docs/concepts/storage/persistent-volumes
FIELDS:
apiVersion <string> #api版本
APIVersion defines the versioned schema of this representation of an
object. Servers should convert recognized schemas to the latest internal
value, and may reject unrecognized values. More info:
https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
kind <string> #类型
Kind is a string value representing the REST resource this object
represents. Servers may infer this from the endpoint the client submits
requests to. Cannot be updated. In CamelCase. More info:
https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
metadata <Object> #元数据
Standard object's metadata. More info:
https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata
spec <Object> #用于定义pv的容量，访问模式和回收策略的
Spec defines a specification of a persistent volume owned by the cluster.
Provisioned by an administrator. More info:
https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistent-volumes
status <Object> #状态
Status represents the current information/status for the persistent volume.
Populated by the system. Read-only. More info:
https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistent-volumes
查看PV spec的几个通用字段，了解PV对存储系统的支持可通过哪些插件来实现
[root@k8s-master1 ~]# kubectl explain pv.spec
KIND: PersistentVolume
VERSION: v1
RESOURCE: spec <Object>
DESCRIPTION:
Spec defines a specification of a persistent volume owned by the cluster.
Provisioned by an administrator. More info:
https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistent-volumes
PersistentVolumeSpec is the specification of a persistent volume.
FIELDS:
accessModes <[]string>
#访问模式，每个 PV 的访问模式都设置为特定卷支持的特定模式。
AccessModes contains all ways the volume can be mounted. More info:
https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes
awsElasticBlockStore <Object>
AWSElasticBlockStore represents an AWS Disk resource that is attached to a
kubelet's host machine and then exposed to the pod. More info:
https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore
azureDisk <Object>
AzureDisk represents an Azure Data Disk mount on the host and bind mount to
the pod.
azureFile <Object>
AzureFile represents an Azure File Service mount on the host and bind mount
to the pod.
capacity <map[string]string>
#PV 将具有特定的存储容量。目前，存储大小是唯一可以设置或请求的资源。未来的属性可能包括 IOPS、吞吐量等。
A description of the persistent volume's resources and capacity. More info:
https://kubernetes.io/docs/concepts/storage/persistent-volumes#capacity
cephfs <Object>
CephFS represents a Ceph FS mount on the host that shares a pod's lifetime
cinder <Object>
Cinder represents a cinder volume attached and mounted on kubelets host
machine. More info: https://examples.k8s.io/mysql-cinder-pd/README.md
claimRef <Object>
ClaimRef is part of a bi-directional binding between PersistentVolume and
PersistentVolumeClaim. Expected to be non-nil when bound. claim.VolumeName
is the authoritative bind between PV and PVC. More info:
https://kubernetes.io/docs/concepts/storage/persistent-volumes#binding
csi <Object>
CSI represents storage that is handled by an external CSI driver (Beta
feature).
fc <Object>
FC represents a Fibre Channel resource that is attached to a kubelet's host
machine and then exposed to the pod.
flexVolume <Object>
FlexVolume represents a generic volume resource that is
provisioned/attached using an exec based plugin.
flocker <Object>
Flocker represents a Flocker volume attached to a kubelet's host machine
and exposed to the pod for its usage. This depends on the Flocker control
service being running
gcePersistentDisk <Object>
GCEPersistentDisk represents a GCE Disk resource that is attached to a
kubelet's host machine and then exposed to the pod. Provisioned by an
admin. More info:
https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk
glusterfs <Object>
Glusterfs represents a Glusterfs volume that is attached to a host and
exposed to the pod. Provisioned by an admin. More info:
https://examples.k8s.io/volumes/glusterfs/README.md
hostPath <Object>
HostPath represents a directory on the host. Provisioned by a developer or
tester. This is useful for single-node development and testing only!
On-host storage is not supported in any way and WILL NOT WORK in a
multi-node cluster. More info:
https://kubernetes.io/docs/concepts/storage/volumes#hostpath
iscsi <Object>
ISCSI represents an ISCSI Disk resource that is attached to a kubelet's
host machine and then exposed to the pod. Provisioned by an admin.
local <Object>
Local represents directly-attached storage with node affinity
mountOptions <[]string> #挂载选项组成的列表，如：ro,soft和hard等
A list of mount options, e.g. ["ro", "soft"]. Not validated - mount will
simply fail if one is invalid. More info:
https://kubernetes.io/docs/concepts/storage/persistent-volumes/#mount-options
nfs <Object>
NFS represents an NFS mount on the host. Provisioned by an admin. More
info: https://kubernetes.io/docs/concepts/storage/volumes#nfs
nodeAffinity <Object>
#节点亲和性，PV 可以指定节点亲和性来定义限制该卷可以从哪些节点访问的约束。使用 PV 的 Pod 只会被调度到由节点亲和性选择的节点
NodeAffinity defines constraints that limit what nodes this volume can be
accessed from. This field influences the scheduling of pods that use this
volume.
persistentVolumeReclaimPolicy <string>
#PV空间被释放时的处理机制；
What happens to a persistent volume when released from its claim. Valid
options are Retain (default for manually created PersistentVolumes), Delete
(default for dynamically provisioned PersistentVolumes), and Recycle
(deprecated). Recycle must be supported by the volume plugin underlying
this PersistentVolume. More info:
https://kubernetes.io/docs/concepts/storage/persistent-volumes#reclaiming
photonPersistentDisk <Object>
PhotonPersistentDisk represents a PhotonController persistent disk attached
and mounted on kubelets host machine
portworxVolume <Object>
PortworxVolume represents a portworx volume attached and mounted on
kubelets host machine
quobyte <Object>
Quobyte represents a Quobyte mount on the host that shares a pod's lifetime
rbd <Object>
RBD represents a Rados Block Device mount on the host that shares a pod's
lifetime. More info: https://examples.k8s.io/volumes/rbd/README.md
scaleIO <Object>
ScaleIO represents a ScaleIO persistent volume attached and mounted on
Kubernetes nodes.
storageClassName <string>
#当前PV所属的StorageClass的名称；默认为空值，即不属于任何StorageClass。特定类的 PV 只能绑定到请求该类的 PVC
Name of StorageClass to which this persistent volume belongs. Empty value
means that this volume does not belong to any StorageClass.
storageos <Object>
StorageOS represents a StorageOS volume that is attached to the kubelet's
host machine and mounted into the pod More info:
https://examples.k8s.io/volumes/storageos/README.md
volumeMode <string>
#卷模型，用于指定此卷可被用作文件系统还是裸格式的块设备；默认为Filesystem
volumeMode defines if a volume is intended to be used with a formatted
filesystem or to remain in raw block state. Value of Filesystem is implied
when not included in spec.
vsphereVolume <Object>
VsphereVolume represents a vSphere volume attached and mounted on kubelets
host machine
其中，accessModes和persistentVolumeReclaimPolicy字段含义具体如下：
1）accessModes <[]string> #访问模式，每个 PV 的访问模式都设置为特定卷支持的特定模式。
例如，NFS 可以支持多个读/写客户端，但特定的 NFS PV 可能在服务器上导出为只读。每个 PV 都有自己的一组访问模式来描述特定 PV 的功能。
常用的访问模式有：
ReadWriteOnce：可以由单个节点以读写方式挂载;
ReadOnlyMany：可被多个节点同时以只读方式挂载;
ReadWriteMany：可被多个节点同时以读写方式挂载;
ReadWriteOncePod：可以由单个 Pod 以读写方式挂载
2）persistentVolumeReclaimPolicy # PV空间被释放时的处理机制。
可用的类型仅为Retain(默认),Recycle或Delete;
Retain:保持不动，由管理员随后手动回收；
Recycle:空间回收，即删除存储卷目录下所有的文件（包括子目录和隐藏文件），目前仅NFS和hostPath支持此操作;【已废弃】
Delete:删除存储卷，仅部分云端存储系统支持，如：AWS EBS、GCE PD、Azure Disk和cinder，默认是删除动态创建的pv
2. 创建PV资源
下面的资源清单配置示例中定义了一个使用NFS存储后端的PV，支持多路的读写操作。待后端存储系统满足要求时，即可进行如下PV资源的创建：
1）创建nfs共享目录
在NFS服务器上创建NFS需要的共享文件系统
[root@k8s-master1 ~]# mkdir pv
[root@k8s-master1 ~]# cd pv
[root@k8s-master1 pv]# mkdir /data/volume_test/v{1,2,3,4,5,6,7,8,9,10} -p
[root@k8s-master1 pv]# vim /etc/exports #配置共享目录
[root@k8s-master1 pv]# cat /etc/exports
/data/volumes 10.0.0.131/24(rw,no_root_squash)
/data/volume_test/v1 10.0.0.131/24(rw,no_root_squash)
/data/volume_test/v2 10.0.0.131/24(rw,no_root_squash)
/data/volume_test/v3 10.0.0.131/24(rw,no_root_squash)
/data/volume_test/v3 10.0.0.131/24(rw,no_root_squash)
/data/volume_test/v4 10.0.0.131/24(rw,no_root_squash)
/data/volume_test/v5 10.0.0.131/24(rw,no_root_squash)
/data/volume_test/v6 10.0.0.131/24(rw,no_root_squash)
/data/volume_test/v7 10.0.0.131/24(rw,no_root_squash)
/data/volume_test/v8 10.0.0.131/24(rw,no_root_squash)
/data/volume_test/v9 10.0.0.131/24(rw,no_root_squash)
[root@k8s-master1 pv]# exportfs -arv #是配置生效
exportfs: duplicated export entries:
exportfs: 10.0.0.131/24:/data/volume_test/v3
exportfs: 10.0.0.131/24:/data/volume_test/v3
exporting 10.0.0.131/24:/data/volume_test/v9
exporting 10.0.0.131/24:/data/volume_test/v8
exporting 10.0.0.131/24:/data/volume_test/v7
exporting 10.0.0.131/24:/data/volume_test/v6
exporting 10.0.0.131/24:/data/volume_test/v5
exporting 10.0.0.131/24:/data/volume_test/v4
exporting 10.0.0.131/24:/data/volume_test/v3
exporting 10.0.0.131/24:/data/volume_test/v2
exporting 10.0.0.131/24:/data/volume_test/v1
exporting 10.0.0.131/24:/data/volumes
[root@k8s-master1 pv]#
2）编写PV资源配置清单，创建PV资源
[root@k8s-master1 pv]# vim pv-demo.yaml
[root@k8s-master1 pv]# cat pv-demo.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: v1
spec:
capacity:
storage: 1Gi #pv存储空间容量
accessModes: ["ReadWriteOnce"] #访问模式
nfs:
path: /data/volume_test/v1 #使用nfs存储后端的共享目录创建pv
server: 10.0.0.131 #nfs服务器地址
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: v2
spec:
persistentVolumeReclaimPolicy: Delete
capacity:
storage: 2Gi
accessModes: ["ReadWriteMany"]
nfs:
path: /data/volume_test/v2
server: 10.0.0.131
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: v3
spec:
capacity:
storage: 3Gi
accessModes: ["ReadOnlyMany"]
nfs:
path: /data/volume_test/v3
server: 10.0.0.131
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: v4
spec:
capacity:
storage: 4Gi
accessModes: ["ReadWriteOnce","ReadWriteMany"]
nfs:
path: /data/volume_test/v4
server: 10.0.0.131
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: v5
spec:
capacity:
storage: 5Gi
accessModes: ["ReadWriteOnce","ReadWriteMany"]
nfs:
path: /data/volume_test/v5
server: 10.0.0.131
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: v6
spec:
capacity:
storage: 6Gi
accessModes: ["ReadWriteOnce","ReadWriteMany"]
nfs:
path: /data/volume_test/v6
server: 10.0.0.131
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: v7
spec:
capacity:
storage: 7Gi
accessModes: ["ReadWriteOnce","ReadWriteMany"]
nfs:
path: /data/volume_test/v7
server: 10.0.0.131
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: v8
spec:
capacity:
storage: 8Gi
accessModes: ["ReadWriteOnce","ReadWriteMany"]
nfs:
path: /data/volume_test/v8
server: 10.0.0.131
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: v9
spec:
capacity:
storage: 9Gi
accessModes: ["ReadWriteOnce","ReadWriteMany"]
nfs:
path: /data/volume_test/v9
server: 10.0.0.131
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: v10
spec:
capacity:
storage: 10Gi
accessModes: ["ReadWriteOnce","ReadWriteMany"]
nfs:
path: /data/volume_test/v10
server: 10.0.0.131
[root@k8s-master1 pv]# kubectl apply -f pv-demo.yaml
persistentvolume/v1 created
persistentvolume/v2 created
persistentvolume/v3 created
persistentvolume/v4 created
persistentvolume/v5 created
persistentvolume/v6 created
persistentvolume/v7 created
persistentvolume/v8 created
persistentvolume/v9 created
persistentvolume/v10 created
[root@k8s-master1 pv]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
v1 1Gi RWO Retain Available 24s
v10 10Gi RWO,RWX Retain Available 24s
v2 2Gi RWX Delete Available 24s
v3 3Gi ROX Retain Available 24s
v4 4Gi RWO,RWX Retain Available 24s
v5 5Gi RWO,RWX Retain Available 24s
v6 6Gi RWO,RWX Retain Available 24s
v7 7Gi RWO,RWX Retain Available 24s
v8 8Gi RWO,RWX Retain Available 24s
v9 9Gi RWO,RWX Retain Available 24s
使用资源的查看命令可列出PV资源的相关信息，创建完成的PV资源可能处于下列四种状态中的某一种，它们代表PV资源的生命周期中的各个阶段。
Available：可用状态的自由资源，尚未被PVC绑定
Bound：已经绑定到PVC
Release：绑定的PVC已经被删除，但是资源尚未被集群回收
Failed：因自动回收资源失败而处于故障状态
三、创建PVC
PVC是存储卷类型的资源，它通过申请占用某个PV而创建，它与PV是一对一的关系，用户无须关心底层实现细节。申请时，用户只需要指定目标空间的大小，访问模式、PV标签选择器和StorageClass等相关信息即可。
1. 查看PVC的spec字段的可嵌套字段
[root@k8s-master1 ~]# kubectl explain pvc.spec
KIND: PersistentVolumeClaim
VERSION: v1
RESOURCE: spec <Object>
DESCRIPTION:
Spec defines the desired characteristics of a volume requested by a pod
author. More info:
https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims
PersistentVolumeClaimSpec describes the common attributes of storage
devices and allows a Source for provider-specific attributes
FIELDS:
accessModes <[]string> #当前pvc的访问模式，其可用模式与PV相同
AccessModes contains the desired access modes the volume should have. More
info:
https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1
dataSource <Object>
This field can be used to specify either: * An existing VolumeSnapshot
object (snapshot.storage.k8s.io/VolumeSnapshot) * An existing PVC
(PersistentVolumeClaim) * An existing custom resource that implements data
population (Alpha) In order to use custom resource types that implement
data population, the AnyVolumeDataSource feature gate must be enabled. If
the provisioner or an external controller can support the specified data
source, it will create a new volume based on the contents of the specified
data source.
resources <Object>
#当前PVC存储卷需要占用的资源量最小值；目前，PVC的资源限定仅指其空间大小
Resources represents the minimum resources the volume should have. More
info:
https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources
selector <Object>
#绑定时对PV应用的标签选择器（matchLabels）或匹配条件表达式（matchExpressions），用于挑选要绑定的PV；
如果同时指定了两种挑选机制，则必须同时满足两种选择机制的PV才能被选中。
A label query over volumes to consider for binding.
storageClassName <string> #所依赖的存储类的名称
Name of the StorageClass required by the claim. More info:
https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1
volumeMode <string>
#卷模型，用于指定此卷可被用作文件系统还是裸格式的块设备；默认为Filesystem
volumeMode defines what type of volume is required by the claim. Value of
Filesystem is implied when not included in claim spec.
volumeName <string> #用于直接指定要绑定的PV的卷名
VolumeName is the binding reference to the PersistentVolume backing this
claim.
[root@k8s-master1 ~]#
2. 创建PVC资源
下面的配置清单定义了一个PVC资源示例
[root@k8s-master1 pv]# vim pvc-demo1.yaml
You have new mail in /var/spool/mail/root
[root@k8s-master1 pv]# cat pvc-demo1.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: my-pvc1
spec:
accessModes: ["ReadWriteMany"]
resources:
requests:
storage: 2Gi
[root@k8s-master1 pv]# kubectl apply -f pvc-demo1.yaml
persistentvolumeclaim/my-pvc1 created
[root@k8s-master1 pv]# kubectl get pvc -o wide
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE VOLUMEMODE
my-pvc1 Bound v2 2Gi RWX 7s Filesystem
[root@k8s-master1 pv]# kubectl get pv -o wide
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE
v1 1Gi RWO Retain Available 18m Filesystem
v10 10Gi RWO,RWX Retain Available 18m Filesystem
v2 2Gi RWX Delete Bound default/my-pvc1 18m Filesystem
v3 3Gi ROX Retain Available 18m Filesystem
v4 4Gi RWO,RWX Retain Available 18m Filesystem
v5 5Gi RWO,RWX Retain Available 18m Filesystem
v6 6Gi RWO,RWX Retain Available 18m Filesystem
v7 7Gi RWO,RWX Retain Available 18m Filesystem
v8 8Gi RWO,RWX Retain Available 18m Filesystem
v9 9Gi RWO,RWX Retain Available 18m Filesystem
可以看到v2的PV的STATUS是Bound，表示这个pv已经被my-pvc1绑定了，而my-pvc1的pvc绑定到的是v2的pv可使用的容量是2G。
创建好PVC资源后，即可在pod资源中通过persistentVolumeClaim存储卷引用它，而后挂载于容器中进行数据持久化。需要注意的是，PV是集群级别的资源，而PVC则隶属于名称空间，因此，PVC在绑定目标PV时不受名称空间的限制，但是POD引用PVC时，则只能是属于同一名称空间中的资源。
四、在pod中使用PVC
在pod资源中调用PVC资源，只需要在定义volumes时使用persistentVolumeClaim字段嵌套指定两个字段即可：
[root@k8s-master1 ~]# kubectl explain pod.spec.volumes.persistentVolumeClaim
KIND: Pod
VERSION: v1
RESOURCE: persistentVolumeClaim <Object>
DESCRIPTION:
PersistentVolumeClaimVolumeSource represents a reference to a
PersistentVolumeClaim in the same namespace. More info:
https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims
PersistentVolumeClaimVolumeSource references the user's PVC in the same
namespace. This volume finds the bound PV and mounts that volume for the
pod. A PersistentVolumeClaimVolumeSource is, essentially, a wrapper around
another type of volume that is owned by someone else (the system).
FIELDS:
claimName <string> -required- #要调用的PVC存储卷的名称，PVC卷要与pod在同一名称空间中
ClaimName is the name of a PersistentVolumeClaim in the same namespace as
the pod using this volume. More info:
https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims
readOnly <boolean> #是否将存储卷强制挂载为只读模式，默认为false
Will force the ReadOnly setting in VolumeMounts. Default false.
1. 创建pod资源使用PVC
下面的清单定义一个pod资源，直接使用上面创建的pvc资源
[root@k8s-master1 pv]# vim pod-pvc.yaml
You have new mail in /var/spool/mail/root
[root@k8s-master1 pv]# cat pod-pvc.yaml
apiVersion: v1
kind: Pod
metadata:
name: pvc-pod
spec:
containers:
- name: nginx
image: nginx:latest
imagePullPolicy: IfNotPresent
volumeMounts:
- name: html
mountPath: /usr/share/nginx/html
volumes:
- name: html
persistentVolumeClaim:
claimName: my-pvc1
[root@k8s-master1 pv]# kubectl apply -f pod-pvc.yaml
pod/pvc-pod created
[root@k8s-master1 pv]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pvc-pod 1/1 Running 0 77s 10.244.36.75 k8s-node1 <none> <none>
[root@k8s-master1 pv]# kubectl describe pods pvc-pod
Name: pvc-pod
Namespace: default
Priority: 0
Node: k8s-node1/10.0.0.132
Start Time: Sat, 24 Sep 2022 14:08:59 +0800
Labels: <none>
Annotations: cni.projectcalico.org/podIP: 10.244.36.75/32
cni.projectcalico.org/podIPs: 10.244.36.75/32
Status: Running
IP: 10.244.36.75
IPs:
IP: 10.244.36.75
Containers:
nginx:
Container ID: docker://b8152bb121414fed127693d51dd4cead76be69b8a10391be5cc3bfa1db08bfed
Image: nginx:latest
Image ID: docker-pullable://nginx@sha256:b95a99feebf7797479e0c5eb5ec0bdfa5d9f504bc94da550c2f58e839ea6914f
Port: <none>
Host Port: <none>
State: Running
Started: Sat, 24 Sep 2022 14:09:02 +0800
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/usr/share/nginx/html from html (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-5n29f (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
html:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: my-pvc1
ReadOnly: false
default-token-5n29f:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-5n29f
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 100s default-scheduler Successfully assigned default/pvc-pod to k8s-node1
Normal Pulled 97s kubelet Container image "nginx:latest" already present on machine
Normal Created 97s kubelet Created container nginx
Normal Started 97s kubelet Started container nginx
2. 测试数据持久化
资源创建完成后，可以看到pod正处于running状态。登录pod中测试数据的持久化
[root@k8s-master1 pv]# kubectl exec -it pvc-pod -- /bin/sh # cd /usr/share/nginx/html # ls # pwd /usr/share/nginx/html # echo "hello world" >>index.html # cat index.html hello world # exit [root@k8s-master1 pv]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pvc-pod 1/1 Running 0 7m24s 10.244.36.75 k8s-node1 <none> <none> [root@k8s-master1 pv]# curl 10.244.36.75 hello world
删除pod后，查看pvc的变化
[root@k8s-master1 pv]# kubectl delete pods pvc-pod pod "pvc-pod" deleted You have new mail in /var/spool/mail/root [root@k8s-master1 pv]# kubectl get pvc -o wide NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE VOLUMEMODE my-pvc1 Bound v2 2Gi RWX 28m Filesystem # 查看nfs服务器共享目录下是否有之前的数据 [root@k8s-master1 ~]# ll /data/volume_test/v2 total 4 -rw-r--r-- 1 root root 12 Sep 24 14:15 index.html [root@k8s-master1 ~]# cat /data/volume_test/v2/index.html hello world
可以看到pod删除后，pvc仍然与pv绑定，共享目录下的数据依然存在。重新创建pod，依然可以正常访问，实现了数据的持久化
[root@k8s-master1 pv]# kubectl apply -f pod-pvc.yaml pod/pvc-pod created You have new mail in /var/spool/mail/root [root@k8s-master1 pv]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pvc-pod 1/1 Running 0 7s 10.244.36.83 k8s-node1 <none> <none> [root@k8s-master1 pv]# curl 10.244.36.83 hello world [root@k8s-master1 pv]#
若删除pod和pvc后，查看共享目录下的数据是否依然存在
[root@k8s-master1 pv]# kubectl delete pods pvc-pod pod "pvc-pod" deleted You have new mail in /var/spool/mail/root [root@k8s-master1 pv]# kubectl get pods -o wide No resources found in default namespace. [root@k8s-master1 pv]# kubectl delete pvc my-pvc1 persistentvolumeclaim "my-pvc1" deleted [root@k8s-master1 pv]# kubectl get pvc -o wide No resources found in default namespace. [root@k8s-master1 pv]# ls -lrt /data/volume_test/v2/ total 4 -rw-r--r-- 1 root root 12 Sep 24 14:15 index.html [root@k8s-master1 pv]# cat /data/volume_test/v2/index.html hello world [root@k8s-master1 pv]#
可以看到共享目录下的数据依然存在，不会因为pod和pvc的删除导致数据丢失。
3. 查看重建PVC绑定的PV变化
重建pvc，查看之前绑定的pv是否会被再次绑定
[root@k8s-master1 pv]# kubectl apply -f pvc-demo1.yaml persistentvolumeclaim/my-pvc1 created [root@k8s-master1 pv]# kubectl get pvc -o wide NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE VOLUMEMODE my-pvc1 Bound v4 4Gi RWO,RWX 7s Filesystem [root@k8s-master1 pv]# kubectl get pv -o wide NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE v1 1Gi RWO Retain Available 62m Filesystem v10 10Gi RWO,RWX Retain Available 62m Filesystem v2 2Gi RWX Delete Failed default/my-pvc1 62m Filesystem v3 3Gi ROX Retain Available 62m Filesystem v4 4Gi RWO,RWX Retain Bound default/my-pvc1 62m Filesystem v5 5Gi RWO,RWX Retain Available 62m Filesystem v6 6Gi RWO,RWX Retain Available 62m Filesystem v7 7Gi RWO,RWX Retain Available 62m Filesystem v8 8Gi RWO,RWX Retain Available 62m Filesystem v9 9Gi RWO,RWX Retain Available 62m Filesystem
可以看到重建的pvc绑定到了v4的pv上。
删除pvc后，查看pv的状态变化，可以看到v4的pv状态变为Released
[root@k8s-master1 pv]# kubectl delete pvc my-pvc1 persistentvolumeclaim "my-pvc1" deleted You have new mail in /var/spool/mail/root [root@k8s-master1 pv]# kubectl get pv -o wide NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE v1 1Gi RWO Retain Available 64m Filesystem v10 10Gi RWO,RWX Retain Available 64m Filesystem v2 2Gi RWX Delete Failed default/my-pvc1 64m Filesystem v3 3Gi ROX Retain Available 64m Filesystem v4 4Gi RWO,RWX Retain Released default/my-pvc1 64m Filesystem v5 5Gi RWO,RWX Retain Available 64m Filesystem v6 6Gi RWO,RWX Retain Available 64m Filesystem v7 7Gi RWO,RWX Retain Available 64m Filesystem v8 8Gi RWO,RWX Retain Available 64m Filesystem v9 9Gi RWO,RWX Retain Available 64m Filesystem
再次创建pvc，依然不会绑定到状态为Released的PV上，而是绑定了可用的其他pv上
[root@k8s-master1 pv]# kubectl apply -f pvc-demo1.yaml persistentvolumeclaim/my-pvc1 created You have new mail in /var/spool/mail/root [root@k8s-master1 pv]# kubectl get pvc -o wide NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE VOLUMEMODE my-pvc1 Bound v5 5Gi RWO,RWX 5s Filesystem [root@k8s-master1 pv]# kubectl get pv -o wide NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE v1 1Gi RWO Retain Available 68m Filesystem v10 10Gi RWO,RWX Retain Available 68m Filesystem v2 2Gi RWX Delete Failed default/my-pvc1 68m Filesystem v3 3Gi ROX Retain Available 68m Filesystem v4 4Gi RWO,RWX Retain Released default/my-pvc1 68m Filesystem v5 5Gi RWO,RWX Retain Bound default/my-pvc1 68m Filesystem v6 6Gi RWO,RWX Retain Available 68m Filesystem v7 7Gi RWO,RWX Retain Available 68m Filesystem v8 8Gi RWO,RWX Retain Available 68m Filesystem v9 9Gi RWO,RWX Retain Available 68m Filesystem [root@k8s-master1 pv]# kubectl get pvc -o wide NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE VOLUMEMODE my-pvc1 Bound v5 5Gi RWO,RWX 2m9s Filesystem [root@k8s-master1 pv]#
删除所有的pv和已有的pvc后，重建pv和pvc查看是否pvc是否还会与v2的pv绑定
[root@k8s-master1 pv]# kubectl delete pvc my-pvc1 persistentvolumeclaim "my-pvc1" deleted [root@k8s-master1 pv]# kubectl delete -f pv-demo.yaml persistentvolume "v1" deleted persistentvolume "v2" deleted persistentvolume "v3" deleted persistentvolume "v4" deleted persistentvolume "v5" deleted persistentvolume "v6" deleted persistentvolume "v7" deleted persistentvolume "v8" deleted persistentvolume "v9" deleted persistentvolume "v10" deleted [root@k8s-master1 pv]# kubectl apply -f pv-demo.yaml persistentvolume/v1 created persistentvolume/v2 created persistentvolume/v3 created persistentvolume/v4 created persistentvolume/v5 created persistentvolume/v6 created persistentvolume/v7 created persistentvolume/v8 created persistentvolume/v9 created persistentvolume/v10 created You have new mail in /var/spool/mail/root [root@k8s-master1 pv]# kubectl apply -f pvc-demo1.yaml persistentvolumeclaim/my-pvc1 created [root@k8s-master1 pv]# kubectl get pv -o wide NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE v1 1Gi RWO Retain Available 12s Filesystem v10 10Gi RWO,RWX Retain Available 12s Filesystem v2 2Gi RWX Delete Bound default/my-pvc1 12s Filesystem v3 3Gi ROX Retain Available 12s Filesystem v4 4Gi RWO,RWX Retain Available 12s Filesystem v5 5Gi RWO,RWX Retain Available 12s Filesystem v6 6Gi RWO,RWX Retain Available 12s Filesystem v7 7Gi RWO,RWX Retain Available 12s Filesystem v8 8Gi RWO,RWX Retain Available 12s Filesystem v9 9Gi RWO,RWX Retain Available 12s Filesystem [root@k8s-master1 pv]# kubectl get pvc -o wide NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE VOLUMEMODE my-pvc1 Bound v2 2Gi RWX 11s Filesystem
重建pod，再次访问，查看数据是否依旧存在
[root@k8s-master1 pv]# kubectl apply -f pod-pvc.yaml pod/pvc-pod created You have new mail in /var/spool/mail/root [root@k8s-master1 pv]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pvc-pod 1/1 Running 0 8s 10.244.36.86 k8s-node1 <none> <none> [root@k8s-master1 pv]# curl 10.244.36.86 hello world [root@k8s-master1 pv]#
登录到该pod上，修改访问页面数据，存储目录上的数据也发生了变化
[root@k8s-master1 pv]# kubectl exec -it pvc-pod -- /bin/sh # cd /usr/share/nginx/html # cat index.html hello world # echo "test pvc" >index.html # cat index.html test pvc # exit You have new mail in /var/spool/mail/root [root@k8s-master1 pv]# curl 10.244.36.86 test pvc [root@k8s-master1 pv]# cat /data/volume_test/v2/index.html test pvc [root@k8s-master1 pv]#
4. 使用pvc和pv的注意事项
1）每次创建pvc的时候，需要事先有划分好的pv，这样可能不方便，那么可以在创建pvc的时候直接动态创建一个pv这个存储类，pv事先是不存在的
2）pvc和pv绑定，如果使用默认的回收策略retain，那么删除pvc之后，pv会处于released状态，想要继续使用这个pv，需要手动删除pv，kubectl delete pv pv_name，删除pv，不会删除pv里的数据，当重新创建pvc时还会和这个最匹配的pv绑定，数据还是原来数据，不会丢失。
