Velero云原生的灾难恢复和迁移工具
18. Velero
18.1 Velero概述
1. Velero 是vmware开源的一个云原生的灾难恢复和迁移工具,它本身也是开源的,采用GO语言编写,可以安装的备份、恢复和迁移Kubernetes集群资源数据
2. Velero是西班牙语意思是帆船,非常符合Kubernetes社区的命名风格,Velero的开发公司Heptio,已被VMware收购
3. Velero支持标准的K8S集群,既可以是私有云平台也可以是公有云,除了灾难之外它还能做资源转移,支持把容器应用从一个集群迁移到另一个集群
4. Velero的工作方式就是把kubernetes中的数据备份到对象存储以实现高可用的持久化,默认的备份保存时间为720小时,并在需要的时候进行下载和恢复
18.2 Velero与etcd快照备份的区别
1. etcd快照是全局备份,在即使一个资源对象需要恢复,也是需要做全局恢复到备份的状态,即会影响其它namespace中pod运行服务
2. Velero可以有针对性的备份,比如按照namespace单独备份、只备份单独的资源对象等,在恢复的时候只恢复单独的namespace或资源对象,而不影响其它namespace中pod运行服务
3. Velero支持ceph、oss等对象存储,etcd快照是一个为本地文件
4. Velero支持任务计划实现周期备份,但etcd快照也可以基于cronjob实现
18.3 Velero备份流程
Velero客户端调用Kubernetes API Server创建Backup任务
Backuup 控制器基于watch机制通过API Server获取到备份任务
Backup 控制器开始执行备份动作,其会通过请求API Server获取需要备份的数据
Backup 控制器将获取到的数据备份到指定的对象存储server端
18.4 部署minio 私有共享存储
docker run --name minio \
-p 9000:9000 \
-p 9999:9999 \
-d --restart=always \
-e "MINIO_ROOT_USER=admin" \
-e "MINIO_ROOT_PASSWORD=12345678" \
-v /data/minio/data:/data \
minio/minio server /data \
--console-address '0.0.0.0:9999'
-
登录成功
-
18.5 配置对象存储
-
创建桶
-
这就创建好了
18.5 部署Velero
root@deploy-harbor:/# wget https://github.com/vmware-tanzu/velero/releases/download/v1.8.1/velero-v1.8.1-linux-amd64.tar.gz
root@deploy-harbor:/velero# tar xvf velero-v1.8.1-linux-amd64.tar.gz -C /velero
root@deploy-harbor:/velero# cp velero-v1.8.1-linux-amd64/velero /usr/local/bin/
root@deploy-harbor:/velero# velero --help
18.6 配置velero认证环境
#工作目录:
root@deploy-harbor:~# mkdir /velero -p
#认证文件:
root@k8s-master1:/velero# vim velero-auth.txt
#aws用户 这个key不能变value可以变 是minio的账号密码
[default]
aws_access_key_id = admin
aws_secret_access_key = 12345678
18.7 准备user-csr文件
root@deploy-harbor:/velero# vim awsuser-csr.json
{
"CN": "awsuser",
"hosts": [],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "BeiJing",
"L": "BeiJing",
"O": "k8s",
"OU": "System"
}
]
}
18.8 准备证书签发环境
root@deploy-harbor:/velero# apt install golang-cfssl
#方法一 下载改名使用
root@deploy-harbor:/velero# wget https://github.com/cloudflare/cfssl/releases/download/v1.6.1/cfssl_1.6.1_linux_amd64
root@deploy-harbor:/velero# wget https://github.com/cloudflare/cfssl/releases/download/v1.6.1/cfssljson_1.6.1_linux_amd64
root@deploy-harbor:/velero# wget https://github.com/cloudflare/cfssl/releases/download/v1.6.1/cfssl-certinfo_1.6.1_linux_amd64
root@deploy-harbor:/velero# mv cfssl-certinfo_1.6.1_linux_amd64 cfssl-certinfo
root@deploy-harbor:/velero# mv cfssl_1.6.1_linux_amd64 cfssl
root@deploy-harbor:/velero# mv cfssljson_1.6.1_linux_amd64 cfssljson
root@deploy-harbor:/velero# cp cfssl* /usr/local/bin/
root@deploy-harbor:/velero# chmod a+x /usr/local/bin/cfssl*
#方法二 使用/etc/kubeasz
/etc/kubeasz/bin/提供了这些命令拷贝即可
root@deploy-harbor:/velero# cp /etc/kubeasz/bin/cfssl* /usr/local/bin
root@deploy-harbor:/velero# chmod a+x /usr/local/bin/cfssl*
root@deploy-harbor:/velero# cp /etc/kubeasz/clusters/k8s-cluster1/ssl/ca-config.json /velero
18.9 执行证书签发
root@deploy-harbor:/velero# ll
total 20
drwxr-xr-x 2 root root 4096 Nov 23 13:14 ./
drwxr-xr-x 24 root root 4096 Nov 23 12:24 ../
#证书签发文件
-rw-r--r-- 1 root root 220 Nov 23 12:25 awsuser-csr.json
#ca证书配置文件
-rw-r--r-- 1 root root 483 Nov 23 13:13 ca-config.json
#minio认证文件
-rw-r--r-- 1 root root 89 Nov 23 12:25 velero-auth.txt
#源执行命令
1.24.x:
root@deploy-harbor:/velero# /usr/local/bin/cfssl gencert -ca=/etc/kubernetes/ssl/ca.pem -ca-key=/etc/kubernetes/ssl/ca-key.pem -config=/etc/kubeasz/clusters/k8s-cluster1/ssl/ca-config.json -profile=kubernetes ./awsuser-csr.json | cfssljson -bare awsuser
1.23.x:
#添加了修改后 就改了一下路径
root@deploy-harbor:/velero# /usr/local/bin/cfssl gencert -ca=/etc/kubernetes/ssl/ca.pem -ca-key=/etc/kubernetes/ssl/ca-key.pem -config=./ca-config.json -profile=kubernetes ./awsuser-csr.json | cfssljson -bare awsuser
2022/11/23 13:40:18 [INFO] generate received request
2022/11/23 13:40:18 [INFO] received CSR
2022/11/23 13:40:18 [INFO] generating key: rsa-2048
2022/11/23 13:40:18 [INFO] encoded CSR
2022/11/23 13:40:18 [INFO] signed certificate with serial number 204902735149897151390216548318080805156194245414
2022/11/23 13:40:18 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
#生成的证书文件
root@deploy-harbor:/velero# ll
total 32
drwxr-xr-x 2 root root 4096 Nov 23 13:40 ./
drwxr-xr-x 21 root root 4096 Nov 23 13:23 ../
-rw-r--r-- 1 root root 997 Nov 23 13:40 awsuser.csr
-rw-r--r-- 1 root root 220 Nov 23 13:23 awsuser-csr.json
#生成api-server证书路径
-rw------- 1 root root 1679 Nov 23 13:40 awsuser-key.pem
#生成api-server证书路径
-rw-r--r-- 1 root root 1387 Nov 23 13:40 awsuser.pem
-rw-r--r-- 1 root root 483 Nov 23 13:25 ca-config.json
-rw-r--r-- 1 root root 89 Nov 23 13:23 velero-auth.txt
#分发证书到api-server证书路径:
root@deploy-harbor:/velero# cp awsuser-key.pem /etc/kubernetes/ssl/
root@deploy-harbor:/velero# cp awsuser.pem /etc/kubernetes/ssl/
18.10 生成集群认证config文件
#这个IP地址可以是VIP 我这里就指定了master的IP地址了
root@deploy-harbor:/velero# export KUBE_APISERVER="https://192.168.1.70:6443"
root@deploy-harbor:/velero# kubectl config set-cluster kubernetes \
--certificate-authority=/etc/kubernetes/ssl/ca.pem \
--embed-certs=true \
--server=${KUBE_APISERVER} \
--kubeconfig=./awsuser.kubeconfig
#输出
Cluster "kubernetes" set.
root@deploy-harbor:/velero# ll
total 36
drwxr-xr-x 2 root root 4096 Nov 23 13:47 ./
drwxr-xr-x 21 root root 4096 Nov 23 13:23 ../
-rw-r--r-- 1 root root 997 Nov 23 13:40 awsuser.csr
-rw-r--r-- 1 root root 220 Nov 23 13:23 awsuser-csr.json
-rw------- 1 root root 1679 Nov 23 13:40 awsuser-key.pem
#会生成这个证书文件
-rw------- 1 root root 1938 Nov 23 13:47 awsuser.kubeconfig
-rw-r--r-- 1 root root 1387 Nov 23 13:40 awsuser.pem
-rw-r--r-- 1 root root 483 Nov 23 13:25 ca-config.json
-rw-r--r-- 1 root root 89 Nov 23 13:23 velero-auth.txt
root@deploy-harbor:/velero# cat awsuser.kubeconfig
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURsRENDQW55Z0F3SUJBZ0lVTUV2SXV3UXdTL3lDMXFyUlFpZ1duVjYwRXNJd0RRWUpLb1pJaHZjTkFRRUwKQlFBd1lURUxNQWtHQTFVRUJoTUNRMDR4RVRBUEJnTlZCQWdUQ0VoaGJtZGFhRzkxTVFzd0NRWURWUVFIRXdKWQpVekVNTUFvR0ExVUVDaE1EYXpoek1ROHdEUVlEVlFRTEV3WlRlWE4wWlcweEV6QVJCZ05WQkFNVENtdDFZbVZ5CmJtVjBaWE13SUJjTk1qSXhNVEU1TURZME1qQXdXaGdQTWpFeU1qRXdNall3TmpReU1EQmFNR0V4Q3pBSkJnTlYKQkFZVEFrTk9NUkV3RHdZRFZRUUlFd2hJWVc1bldtaHZkVEVMTUFrR0ExVUVCeE1DV0ZNeEREQUtCZ05WQkFvVApBMnM0Y3pFUE1BMEdBMVVFQ3hNR1UzbHpkR1Z0TVJNd0VRWURWUVFERXdwcmRXSmxjbTVsZEdWek1JSUJJakFOCkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQXQ1QkZtZkh4KzhCQXQ1OTg2QnBjV2Z0Q3VYcVoKMzlmMFBNaGhDTXNGVWtjYThjNzFxV3R0b0g4Mmk4UEpxTzdRL1FRbWF0MzdmOEdSZmkxNXd0Z0xBYUtmUXlRdgpGYlZvaks0NDhhNHRtSENtbk0zU1dvWStTU2xmbHMrYXM2cEc4SGVnSmE4WVFvR0d5bmtsb2F3dG0wVEVNcmtPCnd5MG5lbDRMcnl0TTF4QnhXek4wbGsrRlhId1o2NEkrR3JvQ2dvSm91RGtaMTNmcENycjlHUGpRelBpNE1CeVQKeXVaREFLMEE1V3FEcVhxSy9kRVJPb09yVXRoK3ROSHJCNTFqZ0QzQlJFVnZSLytBcG52VFRBM212Ym1UQVdudwo2SGVZZkdtSkpNWDRNZVBCYUIzYjVXa044cWJqckhVVXhXSmZPZG96UG1iQ3FRRmplcWRNVDh4VWZ3SURBUUFCCm8wSXdRREFPQmdOVkhROEJBZjhFQkFNQ0FRWXdEd1lEVlIwVEFRSC9CQVV3QXdFQi96QWRCZ05WSFE0RUZnUVUKYk54STNOS05GeERBa1o0eUJWR1RSMmdOL2ZBd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFMSTdNaUxZN0ZQawpzQ0E2b3hBVkxMZWM0Ry9lQ0wxZUJjNVVkcUdDY2RYbXQ0U2RFWTE3WWRKZ2huV1pqNGRSOTVSVFBscWtPS3hYCkI5eCt6SzZYWVVaSlo4aUEvSUtFcHlFMHZYRlBjcmZPaWdzZDdKaEhiUDFuRFpGcEErNy9XaU1DUXlwNXNXRngKTzdmYjdwVko1bnl5RHZEdHNCY0VFYW5NUHhmKzNwZG83NHFBdWowYjNsNHJyU1RjOTdOS0xyekwyQnppaEI3UQpvaGpiSHpuR1lTQkdRU2IvOW5ZU24rRzBVTG50SlFCRU8xbGdTd0RuMzJNNHd4aThUODJ5bldmYUNHaUpMOWg5CkF2U05XRVVhZmF6UDFPVGFydng3N0xnbnA4SUtGejZ2UzJSVUZJSEl4cTJrQnl4SkE4NzhTMnRTWGROWC9SSlIKam9aaXFuRWZxR1U9Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
server: https://192.168.1.70:6443
name: kubernetes
contexts: null
current-context: ""
kind: Config
preferences: {}
#这里目前是空的用户认证
users: null
- 设置客户端证书认证
root@deploy-harbor:/velero# kubectl config set-credentials awsuser \
--client-certificate=/etc/kubernetes/ssl/awsuser.pem \
--client-key=/etc/kubernetes/ssl/awsuser-key.pem \
--embed-certs=true \
--kubeconfig=./awsuser.kubeconfig
运行上面的认证 这个用户信息就会生成了 生成的是awsuser
私钥:certificate-authority-data
公钥:client-certificate-data
root@deploy-harbor:/velero# cat awsuser.kubeconfig
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURsRENDQW55Z0F3SUJBZ0lVTUV2SXV3UXdTL3lDMXFyUlFpZ1duVjYwRXNJd0RRWUpLb1pJaHZjTkFRRUwKQlFBd1lURUxNQWtHQTFVRUJoTUNRMDR4RVRBUEJnTlZCQWdUQ0VoaGJtZGFhRzkxTVFzd0NRWURWUVFIRXdKWQpVekVNTUFvR0ExVUVDaE1EYXpoek1ROHdEUVlEVlFRTEV3WlRlWE4wWlcweEV6QVJCZ05WQkFNVENtdDFZbVZ5CmJtVjBaWE13SUJjTk1qSXhNVEU1TURZME1qQXdXaGdQTWpFeU1qRXdNall3TmpReU1EQmFNR0V4Q3pBSkJnTlYKQkFZVEFrTk9NUkV3RHdZRFZRUUlFd2hJWVc1bldtaHZkVEVMTUFrR0ExVUVCeE1DV0ZNeEREQUtCZ05WQkFvVApBMnM0Y3pFUE1BMEdBMVVFQ3hNR1UzbHpkR1Z0TVJNd0VRWURWUVFERXdwcmRXSmxjbTVsZEdWek1JSUJJakFOCkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQXQ1QkZtZkh4KzhCQXQ1OTg2QnBjV2Z0Q3VYcVoKMzlmMFBNaGhDTXNGVWtjYThjNzFxV3R0b0g4Mmk4UEpxTzdRL1FRbWF0MzdmOEdSZmkxNXd0Z0xBYUtmUXlRdgpGYlZvaks0NDhhNHRtSENtbk0zU1dvWStTU2xmbHMrYXM2cEc4SGVnSmE4WVFvR0d5bmtsb2F3dG0wVEVNcmtPCnd5MG5lbDRMcnl0TTF4QnhXek4wbGsrRlhId1o2NEkrR3JvQ2dvSm91RGtaMTNmcENycjlHUGpRelBpNE1CeVQKeXVaREFLMEE1V3FEcVhxSy9kRVJPb09yVXRoK3ROSHJCNTFqZ0QzQlJFVnZSLytBcG52VFRBM212Ym1UQVdudwo2SGVZZkdtSkpNWDRNZVBCYUIzYjVXa044cWJqckhVVXhXSmZPZG96UG1iQ3FRRmplcWRNVDh4VWZ3SURBUUFCCm8wSXdRREFPQmdOVkhROEJBZjhFQkFNQ0FRWXdEd1lEVlIwVEFRSC9CQVV3QXdFQi96QWRCZ05WSFE0RUZnUVUKYk54STNOS05GeERBa1o0eUJWR1RSMmdOL2ZBd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFMSTdNaUxZN0ZQawpzQ0E2b3hBVkxMZWM0Ry9lQ0wxZUJjNVVkcUdDY2RYbXQ0U2RFWTE3WWRKZ2huV1pqNGRSOTVSVFBscWtPS3hYCkI5eCt6SzZYWVVaSlo4aUEvSUtFcHlFMHZYRlBjcmZPaWdzZDdKaEhiUDFuRFpGcEErNy9XaU1DUXlwNXNXRngKTzdmYjdwVko1bnl5RHZEdHNCY0VFYW5NUHhmKzNwZG83NHFBdWowYjNsNHJyU1RjOTdOS0xyekwyQnppaEI3UQpvaGpiSHpuR1lTQkdRU2IvOW5ZU24rRzBVTG50SlFCRU8xbGdTd0RuMzJNNHd4aThUODJ5bldmYUNHaUpMOWg5CkF2U05XRVVhZmF6UDFPVGFydng3N0xnbnA4SUtGejZ2UzJSVUZJSEl4cTJrQnl4SkE4NzhTMnRTWGROWC9SSlIKam9aaXFuRWZxR1U9Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
server: https://192.168.1.70:6443
name: kubernetes
contexts: null
current-context: ""
kind: Config
preferences: {}
users:
- name: awsuser
user:
client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUQwakNDQXJxZ0F3SUJBZ0lVSStRb0N4ZURaTTdwQTZta0xOcGc1Z3V5V3lZd0RRWUpLb1pJaHZjTkFRRUwKQlFBd1lURUxNQWtHQTFVRUJoTUNRMDR4RVRBUEJnTlZCQWdUQ0VoaGJtZGFhRzkxTVFzd0NRWURWUVFIRXdKWQpVekVNTUFvR0ExVUVDaE1EYXpoek1ROHdEUVlEVlFRTEV3WlRlWE4wWlcweEV6QVJCZ05WQkFNVENtdDFZbVZ5CmJtVjBaWE13SUJjTk1qSXhNVEl6TVRNek5UQXdXaGdQTWpBM01qRXhNVEF4TXpNMU1EQmFNR0l4Q3pBSkJnTlYKQkFZVEFrTk9NUkF3RGdZRFZRUUlFd2RDWldsS2FXNW5NUkF3RGdZRFZRUUhFd2RDWldsS2FXNW5NUXd3Q2dZRApWUVFLRXdOck9ITXhEekFOQmdOVkJBc1RCbE41YzNSbGJURVFNQTRHQTFVRUF4TUhZWGR6ZFhObGNqQ0NBU0l3CkRRWUpLb1pJaHZjTkFRRUJCUUFEZ2dFUEFEQ0NBUW9DZ2dFQkFOODNKRE5JOEtZOGs3cjlPWVhja2JWd1pPZGsKNHZ6eFB2eE1XM0poaXg0TlNaYjlTVzhreHhjRjFsYUZIQ0dSei9TdEZwTDgvTzduR1d5RlJBVXE1K2hqRmJzdgpnOWEydTN3bmtLK3NWUGVUNlVQcGVWY0ROOUVTcFljRk5ERGd5VnQ5WGZxeUVxNkVzcU9DSVNuUXlLTzNFSURBCkdaUXFzaDBjRGYyMXU0V2JjaHBwSjJWeHRIQmhEQzdOSGROK0pmeC9ZeGU5VThSQTVCOHZQZ2g3S2tJcGtQb3MKN2Fob20ydGlwT3pkWUVTODkxZkFJekthb1BpZ0ppUWZBZGVXMUJldGl2akFZK2Z2MEJvRlNFbU5HVWs1c3kyNwo1TnlRU2RuVWJlcHk5SjY4N2hpdThlWE9aaXdsaDcxdVZaM2h0V21hS2RkT20rYlR6cmpPWUxWWER0a0NBd0VBCkFhTi9NSDB3RGdZRFZSMFBBUUgvQkFRREFnV2dNQjBHQTFVZEpRUVdNQlFHQ0NzR0FRVUZCd01CQmdnckJnRUYKQlFjREFqQU1CZ05WSFJNQkFmOEVBakFBTUIwR0ExVWREZ1FXQkJTVWhHR2hGVzNDb2tISnZrNUlFbi9kUGZURQpaREFmQmdOVkhTTUVHREFXZ0JSczNFamMwbzBYRU1DUm5qSUZVWk5IYUEzOThEQU5CZ2txaGtpRzl3MEJBUXNGCkFBT0NBUUVBV3ZMbWlid1NXV0IzYkcrejJqMi9UajY3MkxDc2xsK2JONGw4SzhVdFZyS2lVbTBzakZkaEFrRloKZHFoWi9UQXdlaDJ4U2NXUytaNU5pTkdiUk5IRFdmVXcyeTR3QldnWFFNVktEaUdZcW5iWGFYQmV4dmxpWVRycwp6aC9wS0o3UThxUjJuTVppd0NWTi90dDg2SktOaWU3Zm0rVjdSVms5WUdaZUZxOHFuWFptd09hYnlPV2FkYmY3ClVMSHM5M0dOT1B6OGF3RGNsbkxIaTZIT05YamVlQ1BXVVV5RDFQbTViVzc0ei92Rm5PQm1tdUpyZnlmYUczRE8Kdk9IOU5qellIb3RSWncwNGVQY2Zza2hNSjRwNzJuRXVmaktkdHVETDBocGZtZ0NoWk5nUGdoRkhVcjI0QnFMOQpHbTdqbDRxSGxTNWFoL2E5TXcyekJPMk9nckZtcXc9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBM3pja00wandwanlUdXYwNWhkeVJ0WEJrNTJUaS9QRSsvRXhiY21HTEhnMUpsdjFKCmJ5VEhGd1hXVm9VY0laSFA5SzBXa3Z6ODd1Y1piSVZFQlNybjZHTVZ1eStEMXJhN2ZDZVFyNnhVOTVQcFErbDUKVndNMzBSS2xod1UwTU9ESlczMWQrcklTcm9TeW80SWhLZERJbzdjUWdNQVpsQ3F5SFJ3Ti9iVzdoWnR5R21rbgpaWEcwY0dFTUxzMGQwMzRsL0g5akY3MVR4RURrSHk4K0NIc3FRaW1RK2l6dHFHaWJhMktrN04xZ1JMejNWOEFqCk1wcWcrS0FtSkI4QjE1YlVGNjJLK01CajUrL1FHZ1ZJU1kwWlNUbXpMYnZrM0pCSjJkUnQ2bkwwbnJ6dUdLN3gKNWM1bUxDV0h2VzVWbmVHMWFab3AxMDZiNXRQT3VNNWd0VmNPMlFJREFRQUJBb0lCQVFDYmFBeXdHQXhUOE1ENQo0dXQxbzBkZkIwMTVQT2c4S3NvclpnNFU2SXl1ODVsVW9pdktVZFM2MXJtUHEwVVdxaW9hczVlUDVzdENtV3M5Ci90cUlyb0tmYkx6b2dnQk1NUlQvTDV5d2NrOEZ2OWtjQ3lVTHk1WEUzaktZZXFzSGpMa2tGWGlrM2UzYnkwK0UKUDBsUGo2amhNc2N1bkpBeWREZGx4TzZJd1VvNWpkOVJZTXlhY3dyV2FpTm1QZzB6YjVDVWtxYS9PUGpVamFOMQpKZUJMbGYvd0pnUnZxcXdCVHUwMzBIWGsrV1pQNk1HNTNQak0vd3RHVTZVV2hteWgwZGNmZ1c5MTRtaDdjcG5oCmlENkJaY0k1UlVGM1lSUFFBMVpnbnJSUzU5VThibHZFWG1zcTR3cDJDUFk4cUZFdEt6VHp3WkV2bkdoekNrRVIKNUM0REVaZ0JBb0dCQVBLYlpyQy9mRWxKdEp5QXFMSmVIWHV1a1RiRnJFbFhKbU0zMzRNRVVYc245QlJta210OQpPODdmSnVJemdXQjliV0xJcHJHV2czK2JsVTViejg1ZFRqSjFuUkZWZlRlM3hicXRuT3dZSVpmTXhHTzFkU0pCCmp3NzJyNlNwbmllVUYvd3ZlM1BKRE1BVDYvTjBXMmpvbzdMcTZEdEU2WU8raXR4TmQ5amVNL2ZCQW9HQkFPdUoKc1FnT2VZZU9EMS9iTXNrMjN3OS9DZDd0eGZMRGJPbHNxQTQ1YlpqbEU2OW1LYXVOYndNaWVjSmhIcmFLc3dOagpHT1BseXJ3cHhJTUhKcFhMMlFyczBSVE8yMytCbHlWbTFIeTVxb0R1T0xOaE1QN2hLaFdMSG1LQXlvV0VMQVpBCk9HajhtMG1KU3ZLMnZFT3ZyS09Ba09uMjdYK3FKTXFzVFRlejh4MFpBb0dCQU5keVdnZXNoVnN3TVV5cWxTV1IKbnUwdllaWWFoWjI3MzBOTll4aWV4UjF4cGJoWVByOW1QNjYvVFhDai9Na0xtaENYMEREMVVXSUpjTlpkZ25YcApOb0JwSWtzNmVERDY2b1dWbUQyeVVHTFdYYkdPZTY4b09UczRwMUs3cnMvSEhHWEFaUW0yQ2JTcC9HZi9kUTM5ClN5S2dxZ3U0Yndlb052amRjd2tNaTJvQkFvR0FmbkZtZGZxL1J3UCtRT2s5ZEVOZXI4VDFHWlh2aEFsaE9GWTYKU1ppT3UxdU9tOFJ1YmtmNDVtZmxrWEh0dVBUd2NNc01HNXZLb0FTVUYvc0l2YlczQ0dSbFFaUml4U3BlWjVleQpUbHFscTUwM0Z5VC8xblF0MFc3am11R2sxdFJEaGYrSHlPU0N3SkZSaU1hTWZwR2FUaFBBdDdqMWVtbHVZdGJvCmhSUzNXRkVDZ1lBZ1BWRE1hdnVPaHNwc01XRUJWOFFHZ2dRQ0kwUzVhWUpidWl5UXE2a0tnS0IvU0doNE9tSlQKK2V2Nk9WYVY1NHplRmFVRDRQeUs2OU53RFdJK3Y5cGk5ZDFPRkM2Ym9iT2VMcm12SGhSdERJOVdseVZxTklLNQpuNXZORTMxVlUrVGVMK2lIc3lvNWE2YWZPTVNkL1VPQjJxaXVidFlEai9JcWlETGRrWURLOXc9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=
- 设置上下文参数
root@deploy-harbor:/velero# kubectl config set-context kubernetes \
--cluster=kubernetes \
--user=awsuser \
--namespace=velero-system \
--kubeconfig=./awsuser.kubeconfig
Context "kubernetes" created.
- 设置默认上下文
#设置默认上下文:
root@deploy-harbor:/velero# kubectl config use-context kubernetes --kubeconfig=awsuser.kubeconfig
- k8s集群中创建awsuser账户
root@deploy-harbor:/velero# kubectl create clusterrolebinding awsuser --clusterrole=cluster-admin --user=awsuser
- 创建namespace
root@deploy-harbor:/velero# kubectl create ns velero-system
- 执行安装
#--bucket 执行minio存储桶的名字 写错的话他会找不到
#--secret-file 指定密码文件
#--namespace velero-system如果你需要不同的备份项目中你就在起一个名字aliyun-system 然后http地址在指定过去
root@deploy-harbor:/velero# velero --kubeconfig ./awsuser.kubeconfig \
install \
--provider aws \
--plugins velero/velero-plugin-for-aws:v1.3.1 \
--bucket velerodata \
--secret-file ./velero-auth.txt \
--use-volume-snapshots=false \
--namespace velero-system \
--backup-location-config region=minio,s3ForcePathStyle="true",s3Url=http://192.168.1.75:9000
- 验证安装
所用的镜像 # velero/velero-plugin-for-aws:v1.3.1
root@deploy-harbor:/velero# kubectl describe pod velero-6755cb8697-phfsr -n velero-system
18.11 创建备份
root@deploy-harbor:/velero# kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system calico-kube-controllers-68555f5f97-mww92 1/1 Running 0 31h
kube-system calico-node-hxbbn 0/1 Running 101 (4m53s ago) 8h
kube-system calico-node-m7xwb 0/1 CrashLoopBackOff 101 (4m33s ago) 8h
kubernetes-dashboard dashboard-metrics-scraper-6dfbcf7959-55bkt 1/1 Running 4 (32h ago) 32h
kubernetes-dashboard kubernetes-dashboard-85997c7f79-64sm8 1/1 Running 5 (32h ago) 32h
myserver coredns-57d94f5d84-bjx7d 1/1 Running 0 8h
myserver coredns-57d94f5d84-zbt5x 1/1 Running 0 8h
myserver linux60-tomcat-app1-deployment-595f7ff67c-494xk 1/1 Running 0 7h55m
myserver linux70-nginx-deployment-55dc5fdcf9-h8kp2 1/1 Running 0 7h55m
myserver net-test1 1/1 Running 1 (33h ago) 2d6h
myserver net-test2 1/1 Running 1 (33h ago) 2d7h
myserver net-test3 1/1 Running 1 (33h ago) 2d5h
myserver net-test4 1/1 Running 1 (33h ago) 36h
myserver net-test5 1/1 Running 1 (33h ago) 35h
velero-system velero-858b9459f9-nvbhc 1/1 Running 2 (20m ago) 23m
#velero backup --help
#--include-namespaces 指定名称空间备份
DATE=`date +%Y%m%d%H%M%S`
velero backup create myserver-ns-backup-${DATE} \
--include-namespaces myserver \
--kubeconfig=./awsuser.kubeconfig \
--namespace velero-system
- 备份成功了 针对myserver备份
18.12 恢复数据
- 删除前
#我删除了整个myserver名称空间下的数据 看看是否能恢复回来
root@deploy-harbor:/velero# kubectl delete namespaces myserver
- 恢复-找到你要恢复的名字
根据备份文件所恢复的
#--kubeconfig 这个是指定认证文件 也可以不使用awsuser 使用/root/.kube/config
# --wait 加上这个wait就会同步的模式 不加就是异步 异步就是请求发给api-server就完成了 同步就是必须执行完成后才可以退出
#--namespace velero-system 这个参数不要改这个是指向velero所使用的
velero restore create --from-backup myserver-ns-backup-20221123150613 --wait \
--kubeconfig=./awsuser.kubeconfig \
--namespace velero-system
-
数据一下子就回来了 把所有的资源都恢复了 例如 deployment svc pod 等
-
备份所有资源
#区别在于没有指定固定的--include-namespaces
#myserver-ns-backup这个名字是自己起的
DATE=`date +%Y%m%d%H%M%S`
velero backup create myserver-ns-backup-${DATE} \
--kubeconfig=./awsuser.kubeconfig \
--namespace velero-system
- 根据namespace恢复数据
velero restore create --from-backup myserver-ns-backup-20221123153523 --wait \
--kubeconfig=/root/.kube/config \
--include-cluster-resources=true \
--include-namespaces default \
--namespace velero-system
velero restore create --from-backup myserver-ns-backup-20221123153523 --wait \
--kubeconfig=/root/.kube/config \
--include-cluster-resources=true \
--include-namespaces myserver \
--namespace velero-system
18.13 备份脚本
root@deploy-harbor:/velero# vim ns-backup.sh
#!/bin/bash
NS_NAME=`kubectl get ns | awk '{if (NR>1){print}}' | awk '{print $1}'`
DATE=`date +%Y%m%d%H%M%S`
cd /velero/
for i in $NS_NAME;do
velero backup create ${i}-ns-backup-${DATE} \
--kubeconfig=/root/.kube/config \
--include-cluster-resources=true \
--include-namespaces ${i} \
--namespace velero-system
done
- 成功
18.14 备份指定资源对象
velero backup create pod-backup-xxx \
include-cluster-resources=true \
--ordered-resources \
'pods=myserver/net-test1,defafut/net-test1' \
-- namespace velero-system --include-namespaces=myserver,defafut