基于velero+minio对k8s进行备份和恢复
一、velero介绍
Velero 是由vmware公司团队研发开源工具，用于安全地备份、恢复和迁移 Kubernetes 集群和持久卷。它可以在本地和公共云中运行。Velero 由一个在您的 Kubernetes 集群中作为部署运行的服务器进程和一个命令行界面 (CLI) 组成，DevOps 团队和平台操作员可通过该界面配置计划备份、触发临时备份、执行恢复等。官网地址：https://velero.io/
Velero备份流程
整体架构
二、准备minio存储环境
1、安装minio存储
docker pull minio/minio:latest
docker run -p 9000:9000 -p 9090:9090 \
--net=host \
--name minio \
-d --restart=always \
-e "MINIO_ACCESS_KEY=admin" \
-e "MINIO_SECRET_KEY=minio123" \
-v /home/minio/data:/data \
-v /home/minio/config:/root/.minio \
minio/minio server \
/data --console-address ":9090" -address ":9000"
[root@easzlab-minio-01 ~]# netstat -tnlp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 127.0.0.1:40963 0.0.0.0:* LISTEN 737/containerd tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 669/systemd-resolve tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 830/sshd: /usr/sbin tcp 0 0 127.0.0.1:6010 0.0.0.0:* LISTEN 1437/sshd: root@pts tcp6 0 0 :::9000 :::* LISTEN 5020/minio tcp6 0 0 :::22 :::* LISTEN 830/sshd: /usr/sbin tcp6 0 0 ::1:6010 :::* LISTEN 1437/sshd: root@pts tcp6 0 0 :::9090 :::* LISTEN 5020/minio [root@easzlab-minio-01 ~]#
admin minio123
2、创建velerodata Buckets
3、在master-01节点部署velero
下载安装包 wget https://github.com/vmware-tanzu/velero/releases/download/v1.8.1/velero-v1.8.1-linux-amd64.tar.gz tar -xvzf velero-v1.8.1-linux-amd64.tar.gz mv velero /usr/local/bin
4、配置velero认证环境
mkdir /data/velero -p cd /data/velero root@easzlab-k8s-master-01:/data/velero# cat velero-auth.txt [default] aws_access_key_id = admin aws_secret_access_key = minio123
5、将velero安装到k8s集群
root@easzlab-k8s-master-01:/data/velero# kubectl create ns velero-system namespace/velero-system created root@easzlab-k8s-master-01:/data/velero# velero --kubeconfig /root/.kube/config \ > install \ > --provider aws \ > --plugins velero/velero-plugin-for-aws:v1.3.1 \ > --bucket velerodata \ > --secret-file ./velero-auth.txt \ > --use-volume-snapshots=false \ > --namespace velero-system \ > --backup-location-config region=minio,s3ForcePathStyle="true",s3Url=http://172.16.88.170:9000 CustomResourceDefinition/backups.velero.io: attempting to create resource CustomResourceDefinition/backups.velero.io: attempting to create resource client CustomResourceDefinition/backups.velero.io: created CustomResourceDefinition/backupstoragelocations.velero.io: attempting to create resource CustomResourceDefinition/backupstoragelocations.velero.io: attempting to create resource client CustomResourceDefinition/backupstoragelocations.velero.io: created CustomResourceDefinition/deletebackuprequests.velero.io: attempting to create resource CustomResourceDefinition/deletebackuprequests.velero.io: attempting to create resource client CustomResourceDefinition/deletebackuprequests.velero.io: created CustomResourceDefinition/downloadrequests.velero.io: attempting to create resource CustomResourceDefinition/downloadrequests.velero.io: attempting to create resource client CustomResourceDefinition/downloadrequests.velero.io: created CustomResourceDefinition/podvolumebackups.velero.io: attempting to create resource CustomResourceDefinition/podvolumebackups.velero.io: attempting to create resource client CustomResourceDefinition/podvolumebackups.velero.io: created CustomResourceDefinition/podvolumerestores.velero.io: attempting to create resource CustomResourceDefinition/podvolumerestores.velero.io: attempting to create resource client CustomResourceDefinition/podvolumerestores.velero.io: created CustomResourceDefinition/resticrepositories.velero.io: attempting to create resource CustomResourceDefinition/resticrepositories.velero.io: attempting to create resource client CustomResourceDefinition/resticrepositories.velero.io: created CustomResourceDefinition/restores.velero.io: attempting to create resource CustomResourceDefinition/restores.velero.io: attempting to create resource client CustomResourceDefinition/restores.velero.io: created CustomResourceDefinition/schedules.velero.io: attempting to create resource CustomResourceDefinition/schedules.velero.io: attempting to create resource client CustomResourceDefinition/schedules.velero.io: created CustomResourceDefinition/serverstatusrequests.velero.io: attempting to create resource CustomResourceDefinition/serverstatusrequests.velero.io: attempting to create resource client CustomResourceDefinition/serverstatusrequests.velero.io: created CustomResourceDefinition/volumesnapshotlocations.velero.io: attempting to create resource CustomResourceDefinition/volumesnapshotlocations.velero.io: attempting to create resource client CustomResourceDefinition/volumesnapshotlocations.velero.io: created Waiting for resources to be ready in cluster... Namespace/velero-system: attempting to create resource Namespace/velero-system: attempting to create resource client Namespace/velero-system: already exists, proceeding Namespace/velero-system: created ClusterRoleBinding/velero-velero-system: attempting to create resource ClusterRoleBinding/velero-velero-system: attempting to create resource client ClusterRoleBinding/velero-velero-system: created ServiceAccount/velero: attempting to create resource ServiceAccount/velero: attempting to create resource client ServiceAccount/velero: created Secret/cloud-credentials: attempting to create resource Secret/cloud-credentials: attempting to create resource client Secret/cloud-credentials: created BackupStorageLocation/default: attempting to create resource BackupStorageLocation/default: attempting to create resource client BackupStorageLocation/default: created Deployment/velero: attempting to create resource Deployment/velero: attempting to create resource client Deployment/velero: created Velero is installed! ⛵ Use 'kubectl logs deployment/velero -n velero-system' to view the status. root@easzlab-k8s-master-01:/data/velero#
root@easzlab-k8s-master-01:/data/velero# kubectl get pod -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-kube-controllers-5c8bb696bb-fxbmr 1/1 Running 1 (2d7h ago) 6d18h kube-system calico-node-2qtfm 1/1 Running 1 (2d7h ago) 6d18h kube-system calico-node-8l78t 1/1 Running 1 (2d7h ago) 6d18h kube-system calico-node-9b75m 1/1 Running 1 (2d7h ago) 6d18h kube-system calico-node-k75jh 1/1 Running 1 (2d7h ago) 6d18h kube-system calico-node-kmbhs 1/1 Running 1 (2d7h ago) 6d18h kube-system calico-node-lxfk9 1/1 Running 1 (2d7h ago) 6d18h kube-system coredns-69548bdd5f-6df7j 1/1 Running 1 (2d7h ago) 6d6h kube-system coredns-69548bdd5f-nl5qc 1/1 Running 1 (2d7h ago) 6d6h kubernetes-dashboard dashboard-metrics-scraper-8c47d4b5d-2d275 1/1 Running 1 (2d7h ago) 6d6h kubernetes-dashboard kubernetes-dashboard-5676d8b865-6l8n8 1/1 Running 1 (2d7h ago) 6d6h linux70 linux70-tomcat-app1-deployment-5d666575cc-kbjhk 1/1 Running 1 (2d7h ago) 4d7h myserver linux70-nginx-deployment-55dc5fdcf9-58ll2 1/1 Running 1 (2d7h ago) 4d7h myserver linux70-nginx-deployment-55dc5fdcf9-6xcjk 1/1 Running 1 (2d7h ago) 4d7h myserver linux70-nginx-deployment-55dc5fdcf9-cxg5m 1/1 Running 1 (2d7h ago) 4d7h myserver linux70-nginx-deployment-55dc5fdcf9-gv2gk 1/1 Running 1 (2d7h ago) 4d7h velero-system velero-858b9459f9-c7qc9 1/1 Running 0 2m29s root@easzlab-k8s-master-01:/data/velero#
6、测试备份功能
root@easzlab-k8s-master-01:~# DATE=`date +%Y%m%d%H%M%S`
root@easzlab-k8s-master-01:~# velero backup create default-backup-${DATE} --include-cluster-resources=true --include-namespaces kube-system --kubeconfig=/root/.kube/config --namespace velero-system
Backup request "default-backup-20220804202021" submitted successfully.
Run `velero backup describe default-backup-20220804202021` or `velero backup logs default-backup-20220804202021` for more details.
root@easzlab-k8s-master-01:~#
root@easzlab-k8s-master-01:~# velero backup create myserver-backup-${DATE} --include-cluster-resources=true --include-namespaces myserver --kubeconfig=/root/.kube/config --namespace velero-system
Backup request "myserver-backup-20220804202021" submitted successfully.
Run `velero backup describe myserver-backup-20220804202021` or `velero backup logs myserver-backup-20220804202021` for more details.
root@easzlab-k8s-master-01:~#
7、检查备份是否成功
8、使用脚本备份
root@easzlab-k8s-master-01:~# cat velero-k8s-backup.sh #!/bin/bash NS_NAME=$(kubectl get ns |awk '{if(NR>1){print $1}}') DATE=$(date +%Y%m%d%H%M%S) cd /data/velero/ for i in $NS_NAME;do /usr/local/bin/velero backup create ${i}-ns-backup-${DATE} \ --include-cluster-resources=true \ --include-namespaces ${i} \ --kubeconfig=/root/.kube/config \ --namespace velero-system done root@easzlab-k8s-master-01:~# root@easzlab-k8s-master-01:~# bash velero-k8s-backup.sh Backup request "default-ns-backup-20220805165331" submitted successfully. Run `velero backup describe default-ns-backup-20220805165331` or `velero backup logs default-ns-backup-20220805165331` for more details. Backup request "kube-node-lease-ns-backup-20220805165331" submitted successfully. Run `velero backup describe kube-node-lease-ns-backup-20220805165331` or `velero backup logs kube-node-lease-ns-backup-20220805165331` for more details. Backup request "kube-public-ns-backup-20220805165331" submitted successfully. Run `velero backup describe kube-public-ns-backup-20220805165331` or `velero backup logs kube-public-ns-backup-20220805165331` for more details. Backup request "kube-system-ns-backup-20220805165331" submitted successfully. Run `velero backup describe kube-system-ns-backup-20220805165331` or `velero backup logs kube-system-ns-backup-20220805165331` for more details. Backup request "kubernetes-dashboard-ns-backup-20220805165331" submitted successfully. Run `velero backup describe kubernetes-dashboard-ns-backup-20220805165331` or `velero backup logs kubernetes-dashboard-ns-backup-20220805165331` for more details. Backup request "linux70-ns-backup-20220805165331" submitted successfully. Run `velero backup describe linux70-ns-backup-20220805165331` or `velero backup logs linux70-ns-backup-20220805165331` for more details. Backup request "myserver-ns-backup-20220805165331" submitted successfully. Run `velero backup describe myserver-ns-backup-20220805165331` or `velero backup logs myserver-ns-backup-20220805165331` for more details. Backup request "velero-system-ns-backup-20220805165331" submitted successfully. Run `velero backup describe velero-system-ns-backup-20220805165331` or `velero backup logs velero-system-ns-backup-20220805165331` for more details. root@easzlab-k8s-master-01:~#
9、添加到crontab计划任务
crontab -u root -e
*/5 * * * * /usr/bin/bash /data/velero/velero-k8s-backup.sh >> /data/velero/velero-k8s-backup.log 2>&1 #每五分钟备份一次
[root@easzlab-k8s-master-01 velero]# systemctl status cron.service ● cron.service - Regular background program processing daemon Loaded: loaded (/lib/systemd/system/cron.service; enabled; vendor preset: enabled) Active: active (running) since Sun 2022-11-06 10:54:03 CST; 24min ago Docs: man:cron(8) Main PID: 4174694 (cron) Tasks: 1 (limit: 4612) Memory: 3.9M CGroup: /system.slice/cron.service └─4174694 /usr/sbin/cron -f Nov 06 10:54:03 easzlab-k8s-master-01 cron[4174694]: (CRON) INFO (Skipping @reboot jobs -- not system startup) Nov 06 10:55:01 easzlab-k8s-master-01 CRON[4175195]: (root) CMD (/usr/bin/bash /data/velero/velero-k8s-backup.sh) Nov 06 10:55:01 easzlab-k8s-master-01 CRON[4175187]: (CRON) info (No MTA installed, discarding output) Nov 06 11:00:01 easzlab-k8s-master-01 cron[4174694]: (root) RELOAD (crontabs/root) Nov 06 11:00:01 easzlab-k8s-master-01 CRON[4179501]: (root) CMD (/usr/bin/bash /data/velero/velero-k8s-backup.sh 2>&1) Nov 06 11:01:01 easzlab-k8s-master-01 cron[4174694]: (root) RELOAD (crontabs/root) Nov 06 11:05:01 easzlab-k8s-master-01 CRON[4182083]: (root) CMD (/usr/bin/bash /data/velero/velero-k8s-backup.sh >> /data/velero/velero-k8s-backup.log 2>&1) Nov 06 11:10:01 easzlab-k8s-master-01 CRON[4184741]: (root) CMD (/usr/bin/bash /data/velero/velero-k8s-backup.sh >> /data/velero/velero-k8s-backup.log 2>&1) Nov 06 11:15:01 easzlab-k8s-master-01 CRON[4187297]: (root) CMD (/usr/bin/bash /data/velero/velero-k8s-backup.sh >> /data/velero/velero-k8s-backup.log 2>&1) Nov 06 11:17:01 easzlab-k8s-master-01 CRON[4188387]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) [root@easzlab-k8s-master-01 velero]# [root@easzlab-k8s-master-01 velero]# cat /data/velero/velero-k8s-backup.log Backup request "argocd-ns-backup-20221106111001" submitted successfully. Run `velero backup describe argocd-ns-backup-20221106111001` or `velero backup logs argocd-ns-backup-20221106111001` for more details. Backup request "default-ns-backup-20221106111001" submitted successfully. Run `velero backup describe default-ns-backup-20221106111001` or `velero backup logs default-ns-backup-20221106111001` for more details. Backup request "ingress-nginx-ns-backup-20221106111001" submitted successfully. Run `velero backup describe ingress-nginx-ns-backup-20221106111001` or `velero backup logs ingress-nginx-ns-backup-20221106111001` for more details. Backup request "istio-ns-backup-20221106111001" submitted successfully. Run `velero backup describe istio-ns-backup-20221106111001` or `velero backup logs istio-ns-backup-20221106111001` for more details. Backup request "istio-system-ns-backup-20221106111001" submitted successfully. Run `velero backup describe istio-system-ns-backup-20221106111001` or `velero backup logs istio-system-ns-backup-20221106111001` for more details. Backup request "k8spacket-ns-backup-20221106111001" submitted successfully. Run `velero backup describe k8spacket-ns-backup-20221106111001` or `velero backup logs k8spacket-ns-backup-20221106111001` for more details. Backup request "kube-node-lease-ns-backup-20221106111001" submitted successfully. Run `velero backup describe kube-node-lease-ns-backup-20221106111001` or `velero backup logs kube-node-lease-ns-backup-20221106111001` for more details. Backup request "kube-public-ns-backup-20221106111001" submitted successfully. Run `velero backup describe kube-public-ns-backup-20221106111001` or `velero backup logs kube-public-ns-backup-20221106111001` for more details. Backup request "kube-system-ns-backup-20221106111001" submitted successfully. Run `velero backup describe kube-system-ns-backup-20221106111001` or `velero backup logs kube-system-ns-backup-20221106111001` for more details. Backup request "kubernetes-dashboard-ns-backup-20221106111001" submitted successfully. Run `velero backup describe kubernetes-dashboard-ns-backup-20221106111001` or `velero backup logs kubernetes-dashboard-ns-backup-20221106111001` for more details. Backup request "linux-ns-backup-20221106111001" submitted successfully. Run `velero backup describe linux-ns-backup-20221106111001` or `velero backup logs linux-ns-backup-20221106111001` for more details. Backup request "magedu-ns-backup-20221106111001" submitted successfully. Run `velero backup describe magedu-ns-backup-20221106111001` or `velero backup logs magedu-ns-backup-20221106111001` for more details. Backup request "myserver-ns-backup-20221106111001" submitted successfully. Run `velero backup describe myserver-ns-backup-20221106111001` or `velero backup logs myserver-ns-backup-20221106111001` for more details. Backup request "python-ns-backup-20221106111001" submitted successfully. Run `velero backup describe python-ns-backup-20221106111001` or `velero backup logs python-ns-backup-20221106111001` for more details. Backup request "root-ns-backup-20221106111001" submitted successfully. Run `velero backup describe root-ns-backup-20221106111001` or `velero backup logs root-ns-backup-20221106111001` for more details. Backup request "velero-system-ns-backup-20221106111001" submitted successfully. Run `velero backup describe velero-system-ns-backup-20221106111001` or `velero backup logs velero-system-ns-backup-20221106111001` for more details. Backup request "argocd-ns-backup-20221106111501" submitted successfully. Run `velero backup describe argocd-ns-backup-20221106111501` or `velero backup logs argocd-ns-backup-20221106111501` for more details. Backup request "default-ns-backup-20221106111501" submitted successfully. Run `velero backup describe default-ns-backup-20221106111501` or `velero backup logs default-ns-backup-20221106111501` for more details. Backup request "ingress-nginx-ns-backup-20221106111501" submitted successfully. Run `velero backup describe ingress-nginx-ns-backup-20221106111501` or `velero backup logs ingress-nginx-ns-backup-20221106111501` for more details. Backup request "istio-ns-backup-20221106111501" submitted successfully. Run `velero backup describe istio-ns-backup-20221106111501` or `velero backup logs istio-ns-backup-20221106111501` for more details. Backup request "istio-system-ns-backup-20221106111501" submitted successfully. Run `velero backup describe istio-system-ns-backup-20221106111501` or `velero backup logs istio-system-ns-backup-20221106111501` for more details. Backup request "k8spacket-ns-backup-20221106111501" submitted successfully. Run `velero backup describe k8spacket-ns-backup-20221106111501` or `velero backup logs k8spacket-ns-backup-20221106111501` for more details. Backup request "kube-node-lease-ns-backup-20221106111501" submitted successfully. Run `velero backup describe kube-node-lease-ns-backup-20221106111501` or `velero backup logs kube-node-lease-ns-backup-20221106111501` for more details. Backup request "kube-public-ns-backup-20221106111501" submitted successfully. Run `velero backup describe kube-public-ns-backup-20221106111501` or `velero backup logs kube-public-ns-backup-20221106111501` for more details. Backup request "kube-system-ns-backup-20221106111501" submitted successfully. Run `velero backup describe kube-system-ns-backup-20221106111501` or `velero backup logs kube-system-ns-backup-20221106111501` for more details. Backup request "kubernetes-dashboard-ns-backup-20221106111501" submitted successfully. Run `velero backup describe kubernetes-dashboard-ns-backup-20221106111501` or `velero backup logs kubernetes-dashboard-ns-backup-20221106111501` for more details. Backup request "linux-ns-backup-20221106111501" submitted successfully. Run `velero backup describe linux-ns-backup-20221106111501` or `velero backup logs linux-ns-backup-20221106111501` for more details. Backup request "magedu-ns-backup-20221106111501" submitted successfully. Run `velero backup describe magedu-ns-backup-20221106111501` or `velero backup logs magedu-ns-backup-20221106111501` for more details. Backup request "myserver-ns-backup-20221106111501" submitted successfully. Run `velero backup describe myserver-ns-backup-20221106111501` or `velero backup logs myserver-ns-backup-20221106111501` for more details. Backup request "python-ns-backup-20221106111501" submitted successfully. Run `velero backup describe python-ns-backup-20221106111501` or `velero backup logs python-ns-backup-20221106111501` for more details. Backup request "root-ns-backup-20221106111501" submitted successfully. Run `velero backup describe root-ns-backup-20221106111501` or `velero backup logs root-ns-backup-20221106111501` for more details. Backup request "velero-system-ns-backup-20221106111501" submitted successfully. Run `velero backup describe velero-system-ns-backup-20221106111501` or `velero backup logs velero-system-ns-backup-20221106111501` for more details. [root@easzlab-k8s-master-01 velero]#
crontab -u root -e
#*/5 * * * * /usr/bin/bash /data/velero/velero-k8s-backup.sh >> /data/velero/velero-k8s-backup.log 2>&1 #每五分钟备份一次
5 0 * * * /usr/bin/bash /data/velero/velero-k8s-backup.sh >> /data/velero/velero-k8s-backup.log 2>&1 #每天凌晨05：00备份一次
三、恢复备份测试
1、删除备份pod
root@easzlab-deploy:~/pod-test# kubectl get pod -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-kube-controllers-5c8bb696bb-fxbmr 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-2qtfm 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-8l78t 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-9b75m 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-k75jh 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-kmbhs 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-lxfk9 1/1 Running 1 (2d10h ago) 6d21h kube-system coredns-69548bdd5f-6df7j 1/1 Running 1 (2d10h ago) 6d9h kube-system coredns-69548bdd5f-nl5qc 1/1 Running 1 (2d10h ago) 6d9h kubernetes-dashboard dashboard-metrics-scraper-8c47d4b5d-2d275 1/1 Running 1 (2d10h ago) 6d9h kubernetes-dashboard kubernetes-dashboard-5676d8b865-6l8n8 1/1 Running 1 (2d10h ago) 6d9h linux70 linux70-tomcat-app1-deployment-5d666575cc-kbjhk 1/1 Running 1 (2d10h ago) 4d10h myserver linux70-nginx-deployment-55dc5fdcf9-58ll2 1/1 Running 1 (2d10h ago) 4d10h myserver linux70-nginx-deployment-55dc5fdcf9-6xcjk 1/1 Running 1 (2d10h ago) 4d10h myserver linux70-nginx-deployment-55dc5fdcf9-cxg5m 1/1 Running 1 (2d10h ago) 4d10h myserver linux70-nginx-deployment-55dc5fdcf9-gv2gk 1/1 Running 1 (2d10h ago) 4d10h velero-system velero-858b9459f9-5mxxx 1/1 Running 0 23m root@easzlab-deploy:~/pod-test# kubectl delete -f nginx.yaml deployment.apps "linux70-nginx-deployment" deleted service "linux70-nginx-service" deleted root@easzlab-deploy:~/pod-test# kubectl get pod -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-kube-controllers-5c8bb696bb-fxbmr 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-2qtfm 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-8l78t 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-9b75m 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-k75jh 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-kmbhs 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-lxfk9 1/1 Running 1 (2d10h ago) 6d21h kube-system coredns-69548bdd5f-6df7j 1/1 Running 1 (2d10h ago) 6d9h kube-system coredns-69548bdd5f-nl5qc 1/1 Running 1 (2d10h ago) 6d9h kubernetes-dashboard dashboard-metrics-scraper-8c47d4b5d-2d275 1/1 Running 1 (2d10h ago) 6d9h kubernetes-dashboard kubernetes-dashboard-5676d8b865-6l8n8 1/1 Running 1 (2d10h ago) 6d9h linux70 linux70-tomcat-app1-deployment-5d666575cc-kbjhk 1/1 Running 1 (2d10h ago) 4d10h velero-system velero-858b9459f9-5mxxx 1/1 Running 0 24m root@easzlab-deploy:~/pod-test#
2、使用备份数据进行恢复
root@easzlab-k8s-master-01:~# velero restore create --from-backup myserver-backup-20220804202021 --wait --kubeconfig=/root/.kube/config --namespace velero-system Restore request "myserver-backup-20220804202021-20220804203424" submitted successfully. Waiting for restore to complete. You may safely press ctrl-c to stop waiting - your restore will continue in the background. ..................... Restore completed with status: Completed. You may check for more information using the commands `velero restore describe myserver-backup-20220804202021-20220804203424` and `velero restore logs myserver-backup-20220804202021-20220804203424`. root@easzlab-k8s-master-01:~# kubectl get pod -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-kube-controllers-5c8bb696bb-fxbmr 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-2qtfm 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-8l78t 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-9b75m 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-k75jh 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-kmbhs 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-lxfk9 1/1 Running 1 (2d10h ago) 6d21h kube-system coredns-69548bdd5f-6df7j 1/1 Running 1 (2d10h ago) 6d9h kube-system coredns-69548bdd5f-nl5qc 1/1 Running 1 (2d10h ago) 6d9h kubernetes-dashboard dashboard-metrics-scraper-8c47d4b5d-2d275 1/1 Running 1 (2d10h ago) 6d9h kubernetes-dashboard kubernetes-dashboard-5676d8b865-6l8n8 1/1 Running 1 (2d10h ago) 6d9h linux70 linux70-tomcat-app1-deployment-5d666575cc-kbjhk 1/1 Running 1 (2d10h ago) 4d10h myserver linux70-nginx-deployment-55dc5fdcf9-58ll2 0/1 ContainerCreating 0 36s myserver linux70-nginx-deployment-55dc5fdcf9-6xcjk 1/1 Running 0 36s myserver linux70-nginx-deployment-55dc5fdcf9-cxg5m 0/1 ContainerCreating 0 36s myserver linux70-nginx-deployment-55dc5fdcf9-gv2gk 1/1 Running 0 36s velero-system velero-858b9459f9-5mxxx 1/1 Running 0 25m root@easzlab-k8s-master-01:~# kubectl get pod -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-kube-controllers-5c8bb696bb-fxbmr 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-2qtfm 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-8l78t 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-9b75m 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-k75jh 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-kmbhs 1/1 Running 1 (2d10h ago) 6d21h kube-system calico-node-lxfk9 1/1 Running 1 (2d10h ago) 6d21h kube-system coredns-69548bdd5f-6df7j 1/1 Running 1 (2d10h ago) 6d9h kube-system coredns-69548bdd5f-nl5qc 1/1 Running 1 (2d10h ago) 6d9h kubernetes-dashboard dashboard-metrics-scraper-8c47d4b5d-2d275 1/1 Running 1 (2d10h ago) 6d9h kubernetes-dashboard kubernetes-dashboard-5676d8b865-6l8n8 1/1 Running 1 (2d10h ago) 6d9h linux70 linux70-tomcat-app1-deployment-5d666575cc-kbjhk 1/1 Running 1 (2d10h ago) 4d10h myserver linux70-nginx-deployment-55dc5fdcf9-58ll2 1/1 Running 0 2m6s myserver linux70-nginx-deployment-55dc5fdcf9-6xcjk 1/1 Running 0 2m6s myserver linux70-nginx-deployment-55dc5fdcf9-cxg5m 1/1 Running 0 2m6s myserver linux70-nginx-deployment-55dc5fdcf9-gv2gk 1/1 Running 0 2m6s velero-system velero-858b9459f9-5mxxx 1/1 Running 0 27m root@easzlab-k8s-master-01:~#
3、卸载安装
root@easzlab-k8s-master-01:~# kubectl get pod -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-kube-controllers-5c8bb696bb-fxbmr 1/1 Running 1 (2d7h ago) 6d18h kube-system calico-node-2qtfm 1/1 Running 1 (2d7h ago) 6d18h kube-system calico-node-8l78t 1/1 Running 1 (2d7h ago) 6d18h kube-system calico-node-9b75m 1/1 Running 1 (2d7h ago) 6d18h kube-system calico-node-k75jh 1/1 Running 1 (2d7h ago) 6d18h kube-system calico-node-kmbhs 1/1 Running 1 (2d7h ago) 6d18h kube-system calico-node-lxfk9 1/1 Running 1 (2d7h ago) 6d18h kube-system coredns-69548bdd5f-6df7j 1/1 Running 1 (2d7h ago) 6d7h kube-system coredns-69548bdd5f-nl5qc 1/1 Running 1 (2d7h ago) 6d7h kubernetes-dashboard dashboard-metrics-scraper-8c47d4b5d-2d275 1/1 Running 1 (2d7h ago) 6d7h kubernetes-dashboard kubernetes-dashboard-5676d8b865-6l8n8 1/1 Running 1 (2d7h ago) 6d7h linux70 linux70-tomcat-app1-deployment-5d666575cc-kbjhk 1/1 Running 1 (2d7h ago) 4d7h myserver linux70-nginx-deployment-55dc5fdcf9-58ll2 1/1 Running 1 (2d7h ago) 4d7h myserver linux70-nginx-deployment-55dc5fdcf9-6xcjk 1/1 Running 1 (2d7h ago) 4d7h myserver linux70-nginx-deployment-55dc5fdcf9-cxg5m 1/1 Running 1 (2d7h ago) 4d7h myserver linux70-nginx-deployment-55dc5fdcf9-gv2gk 1/1 Running 1 (2d7h ago) 4d7h velero-system velero-858b9459f9-c7qc9 1/1 Running 0 15m root@easzlab-k8s-master-01:~# root@easzlab-k8s-master-01:~# velero --kubeconfig /root/.kube/config uninstall --namespace velero-system You are about to uninstall Velero. Are you sure you want to continue (Y/N)? y Waiting for velero namespace "velero-system" to be deleted .................................................................... Velero namespace "velero-system" deleted Velero uninstalled ⛵ root@easzlab-k8s-master-01:~# kubectl get pod -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-kube-controllers-5c8bb696bb-fxbmr 1/1 Running 1 (2d9h ago) 6d21h kube-system calico-node-2qtfm 1/1 Running 1 (2d9h ago) 6d21h kube-system calico-node-8l78t 1/1 Running 1 (2d9h ago) 6d21h kube-system calico-node-9b75m 1/1 Running 1 (2d9h ago) 6d21h kube-system calico-node-k75jh 1/1 Running 1 (2d9h ago) 6d21h kube-system calico-node-kmbhs 1/1 Running 1 (2d9h ago) 6d21h kube-system calico-node-lxfk9 1/1 Running 1 (2d9h ago) 6d21h kube-system coredns-69548bdd5f-6df7j 1/1 Running 1 (2d9h ago) 6d9h kube-system coredns-69548bdd5f-nl5qc 1/1 Running 1 (2d9h ago) 6d9h kubernetes-dashboard dashboard-metrics-scraper-8c47d4b5d-2d275 1/1 Running 1 (2d9h ago) 6d9h kubernetes-dashboard kubernetes-dashboard-5676d8b865-6l8n8 1/1 Running 1 (2d9h ago) 6d9h linux70 linux70-tomcat-app1-deployment-5d666575cc-kbjhk 1/1 Running 1 (2d9h ago) 4d9h myserver linux70-nginx-deployment-55dc5fdcf9-58ll2 1/1 Running 1 (2d9h ago) 4d9h myserver linux70-nginx-deployment-55dc5fdcf9-6xcjk 1/1 Running 1 (2d9h ago) 4d9h myserver linux70-nginx-deployment-55dc5fdcf9-cxg5m 1/1 Running 1 (2d9h ago) 4d9h myserver linux70-nginx-deployment-55dc5fdcf9-gv2gk 1/1 Running 1 (2d9h ago) 4d9h root@easzlab-k8s-master-01:~#
