Kubernetes 中进行 PVC 的扩容
允许卷扩展
FEATURE STATE:
Kubernetes v1.11 [beta]PersistentVolume 可以配置为可扩展。将此功能设置为 true 时，允许用户通过编辑相应的 PVC 对象来调整卷大小。
当下层 StorageClass 的 allowVolumeExpansion 字段设置为 true 时，以下类型的卷支持卷扩展。
挂载选项
| 卷类型 | Kubernetes 版本要求 |
|---|---|
| gcePersistentDisk | 1.11 |
| awsElasticBlockStore | 1.11 |
| Cinder | 1.11 |
| glusterfs | 1.11 |
| rbd | 1.11 |
| Azure File | 1.11 |
| Azure Disk | 1.11 |
| Portworx | 1.11 |
| FlexVolume | 1.13 |
| CSI | 1.14 (alpha), 1.16 (beta) |
说明： 此功能仅可用于扩容卷，不能用于缩小卷。
案例一：
先查看storageclass是否配置了动态扩容，主要看storageclass是否存在allowVolumeExpansion字段
[root@192 ~]# kubectl get storageclass default -oyaml apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: default parameters: archiveOnDelete: "false" provisioner: nfs-client-provisioner-default reclaimPolicy: Delete volumeBindingMode: Immediate
可以看到并没有allowVolumeExpansion，此时是不支持动态扩容的，可以扩一下测试看看
[root@192 ~]# kubectl edit pvc -n leijun-mysql comom-store-column-03-24 # Please edit the object below. Lines beginning with a '#' will be ignored, # and an empty file will abort the edit. If an error occurs while saving this file will be # reopened with the relevant failures. # apiVersion: v1 kind: PersistentVolumeClaim metadata: name: comom-store-column-03-24 namespace: default spec: accessModes: - ReadWriteMany dataSource: null resources: requests: storage: 50Gi volumeName: pvc-24fc01a0-6d9b-11ea-9107-b8ca3a62236c status: accessModes: - ReadWriteMany capacity: storage: 10Gi phase: Bound "/tmp/kubectl-edit-fkm33.yaml" 38L, 1180C written error: persistentvolumeclaims "comom-store-column-03-24" could not be patched: persistentvolumeclaims "comom-store-column-03-24" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize You can run `kubectl replace -f /tmp/kubectl-edit-fkm33.yaml` to try this update again.
给storageclass添加allowVolumeExpansion字段
[root@192 ~]# kubectl edit storageclass default # Please edit the object below. Lines beginning with a '#' will be ignored, # and an empty file will abort the edit. If an error occurs while saving this file will be # reopened with the relevant failures. # apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: default parameters: archiveOnDelete: "false" provisioner: nfs-client-provisioner-default reclaimPolicy: Delete volumeBindingMode: Immediate allowVolumeExpansion: true #增加该字段表示允许动态扩容 ~ ~ ~ ~ ~ ~ "/tmp/kubectl-edit-x0fyj.yaml" 24L, 738C written storageclass.storage.k8s.io/default edited
再次修改pvc spec.resources.requests.storage字段
[root@192 ~]# kubectl edit pvc comom-store-column-03-24 spec: accessModes: - ReadWriteMany dataSource: null resources: requests: storage: 30Gi volumeName: pvc-24fc01a0-6d9b-11ea-9107-b8ca3a62236c status: accessModes: - ReadWriteMany capacity: storage: 10Gi phase: Bound "/tmp/kubectl-edit-mibh7.yaml" 38L, 1180C written persistentvolumeclaim/comom-store-column-03-24 edited
案例二：
首先当然是要有一个 Kubernetes 1.11 版本的集群。并且提供了支持 Auto provision 的存储。下面的实验是基于 Azure 的 ACS-Engine 集群。
创建 StorageClass
接下来准备两个 Storage Class 对象，分别命名为 common 和 expend，二者主体基本一致，文件名分别为 sc-common.yaml 以及 sc-exp.yaml：
apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: exp # sc-common.yaml 中这里的值为 common parameters: cachingmode: None kind: Managed storageaccounttype: Standard_LRS provisioner: kubernetes.io/azure-disk reclaimPolicy: Delete volumeBindingMode: Immediate allowVolumeExpansion: true # sc-common.yaml 中删掉这一行
创建一个 PVC
我们接下来创建一个 PVC，初始首先测试一下 common 这个 Storageclass，后续的 PVC 操作都从这一个 YAML 中修改而来。
kind: PersistentVolumeClaim apiVersion: v1 metadata: name: myclaim spec: accessModes: - ReadWriteOnce volumeMode: Filesystem resources: requests: storage: 2Gi storageClassName: common
同样使用 kubectl 创建这个 PVC：
$ kubectl apply -f pvc.yaml persistentvolumeclaim/myclaim created $ kubectl get pvc -w NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE myclaim Pending common 9s myclaim Pending common 10s myclaim Pending pvc-e3f8e886-8776-11e8-b82d-000d3aa2ebc3 0 common 11s myclaim Bound pvc-e3f8e886-8776-11e8-b82d-000d3aa2ebc3 2Gi RWO common 11s
第一次扩容尝试
PVC 进入 Bound 状态之后，我们编辑 pvc.yaml，将容量改成 3Gi，并重新 Apply：
$ cat pvc.yaml | sed "s/2Gi/3Gi/" | kubectl apply -f - Error from server (Forbidden): error when applying patch: ... for: "STDIN": persistentvolumeclaims "myclaim" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
结果表明，这次扩容失败了，失败的原因是 Storageclass 不支持扩容
使用新的 Storageclass 创建 PVC
接下来我们将这个 PVC 删除，使用 exp 这个 Storageclass 重建 PVC：
$ kubectl delete -f pvc.yaml persistentvolumeclaim "myclaim" deleted $ sed -i .bak s/common/exp/ pvc.yaml $ kubectl apply -f pvc.yaml persistentvolumeclaim/myclaim created $ kubectl get pvc -w NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE myclaim Bound pvc-37eb6014-8778-11e8-b82d-000d3aa2ebc3 2Gi RWO exp 11s
创建之后，我们可以再次尝试对 PVC 进行扩容：
$ cat pvc.yaml | sed "s/2Gi/1Gi/" | kubectl apply -f - The PersistentVolumeClaim "myclaim" is invalid: spec.resources.requests.storage: Forbidden: field can not be less than previous value $ cat pvc.yaml | sed "s/2Gi/3Gi/" | kubectl apply -f - persistentvolumeclaim/myclaim configured
这里两次执行命令：
- 缩容是不允许的
- 扩容成功
接下来我们再次获取 PVC 信息：
$ kubectl describe pvc myclaim Name: myclaim Namespace: default StorageClass: exp Status: Bound Volume: pvc-37eb6014-8778-11e8-b82d-000d3aa2ebc3 ... Capacity: 2Gi Access Modes: RWO Conditions: ... FileSystemResizePending ... Waiting for user to (re-)start a pod to finish file system resize of volume on node.
绑定 Pod
新建一个 Deployment 来使用前面创建的 PVC：
apiVersion: extensions/v1beta1 kind: Deployment metadata: name: sleep spec: replicas: 1 template: metadata: labels: app: sleep version: v1 spec: containers: - name: sleep image: dustise/sleep:v0.5 imagePullPolicy: IfNotPresent volumeMounts: - name: data mountPath: "/data" volumes: - name: data persistentVolumeClaim: claimName: myclaim
再次查看 PVC 的情况：
$ kubectl describe pvc myclaim
...
Capacity: 3Gi
容量的修改的确生效了。
绑定之后的 PVC 扩容
再次对这个 PVC 进行扩容，我们这次从 3Gi 扩容到 4Gi：
$ cat pvc.yaml | sed "s/2Gi/4Gi/" | kubectl apply -f -
persistentvolumeclaim/myclaim configured
然后获取一下 PVC 的情况：
$ kubectl describe pvc myclaim
...
Capacity: 3Gi
...
Events:
Warning VolumeResizeFailed 31s (x2 over 56s) volume_expand
Original Error: failed request: autorest/azure: Service returned an error. Status=<nil> Code="OperationNotAllowed" Message="Cannot resize disk k8s-5b49c85f-dynamic-pvc-37eb6014-8778-11e8-b82d-000d3aa2ebc3 while it is attached to running VM /subscriptions/6d9be255-d214-4502-a51d-08e1d9c4a7fb/resourceGroups/k8s/providers/Microsoft.Compute/virtualMachines/k8s-agentpool1-17067717-0."
这一情况看来，这次扩容仍然没有生效，错误信息中有提示，无法在已经成功挂载的卷上进行扩容，因此我们清除所有 Pod：
$ kubectl scale deployment sleep --replicas 0
deployment.extensions/sleep scaled
在相关 Pod 消失之后，我们可以再次 describe pvc myclaim，发现这个 PVC 又一次处于等待绑定的状态中。使用 scale 指令恢复 Deployment 运行：
$ kubectl scale deployment sleep --replicas 1 deployment.extensions/sleep scaled $ kubectl describe pvc myclaim ... Capacity: 4Gi ...
PVC 的扩容再次成功了。
参考：
https://kubernetes.io/zh/docs/concepts/storage/storage-classes/
https://www.cnblogs.com/zphqq/p/13330732.html
https://cloud.tencent.com/developer/article/1469648
