目录
- k8s实战
- 部署harbor作为k8s镜像仓库
- 1.实验目标
- 2.在node1上安装harbor
- 3.编辑harbor配置文件
- 4.执行安装
- 5.浏览器访问
- 6.建立镜像仓库
- 7. 所有节点都配置docker信任harbor仓库并重启docker 注意:所有节点
- 8.docker登陆harbor ( 所有节点 都执行 )
- 9.下载镜像修改tag并push到harbor上 ( 注意:从节点执行 )
- 10.节点上删除镜像
- 11.删除以前的demo项目 注意:主节点执行
- 12.修改demo项目的资源配置清单里的镜像地址
- 13.应用资源配置清单
- 14.报错
- 15.查看docker登陆的密码文件
- 16.将docker密码文件解码成base64编码 解码:base64
- 17.创建并应用docker登陆的Secret资源
- 18.修改demo资源配置清单,添加拉取镜像的参数
- 19.应用资源配置清单并查看
- 20.浏览器查看
- 报错总结:
- 部署harbor作为k8s镜像仓库
k8s实战
部署harbor作为k8s镜像仓库
1.实验目标
部署k8s私有镜像仓库harbor
把demo小项目需要的镜像上传到harbor上
修改demo项目的资源配置清单,镜像地址修改为harbord的地址
2.在node1上安装harbor
[root@node1 ~]# cd /opt/
#上传harbor软件包
[root@node1 /opt]# rz -E
rz waiting to receive.
#解压
[root@node1 /opt]# tar zxf harbor-offline-installer-v1.9.0-rc1.tgz
#进入解压后的文件目录
[root@node1 /opt]# cd harbor/
3.编辑harbor配置文件
#备份
[root@node1 /opt/harbor]# cp harbor.yml harbor.yml.bak
#编辑配置文件
[root@node1 /opt/harbor]# vim harbor.yml
#需要更改的地方
hostname: 10.0.0.11
port: 8888
harbor_admin_password: 123456
data_volume: /data/harbor
4.执行安装
#在安装harbor是许诺先安装docker-compose,否则报错
[root@node1 /opt/harbor]# yum install docker-compose -y
#安装harbor(注意命令执行的所在目录)
[root@node1 /opt/harbor]# ./install.sh
5.浏览器访问
http://10.0.0.11:8888
用户:admin
密码:123456
6.建立镜像仓库
这里有2种访问级别:
公开:任何人都可以直接访问并下载镜像
私有:登陆授权后才允许下载镜像
#注意
如果创建私有仓库,k8s是不能直接下载的,需要配置安全文件
7. 所有节点都配置docker信任harbor仓库并重启docker 注意:所有节点
#配置信任仓库
cat >/etc/docker/daemon.json <<EOF
{
"registry-mirrors": ["https://ig2l319y.mirror.aliyuncs.com"],
"exec-opts": ["native.cgroupdriver=systemd"],
"insecure-registries" : ["http://10.0.0.11:8888"]
}
EOF
#重启docker
systemctl restart docker
###############注意###############
在node1上重启docker后,如果harbor不正常了,重启harbor即可
[root@node1 ~]# cd /opt/harbor
[root@node1 /opt/harbor]# docker-compose restart
Restarting harbor-jobservice ... done
Restarting nginx ... done
Restarting harbor-core ... done
Restarting registryctl ... done
Restarting registry ... done
Restarting harbor-portal ... done
Restarting harbor-db ... done
Restarting redis ... done
Restarting harbor-log ... done
8.docker登陆harbor ( 所有节点 都执行 )
[root@node1 /opt/harbor]# docker login 10.0.0.11:8888
Username: admin
Password: #密码 123456
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
9.下载镜像修改tag并push到harbor上 ( 注意:从节点执行 )
1.在主节点查询镜像存放的节点位置
[root@node1 ~]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
mysql-8fcd9f64-vqkm9 1/1 Running 1 18m 10.2.1.4 node2 <none> <none>
myweb-6f974fdbdc-gsncp 1/1 Running 1 18m 10.2.1.5 node2 <none> <none>
myweb-6f974fdbdc-ngngv 1/1 Running 1 18m 10.2.2.3 node3 <none> <none>
2.根据主节点获取的信息在从节点执行打标签
[root@node2 ~]# docker tag kubeguide/tomcat-app:v1 10.0.0.11:8888/k8s/tomcat-app:v1
[root@node2 ~]# docker tag mysql:5.7 10.0.0.11:8888/k8s/mysql:5.7
3.将打好的标签的镜像上传到harbor仓库
[root@node2 ~]# docker push 10.0.0.11:8888/k8s/tomcat-app:v1
[root@node2 ~]# docker push 10.0.0.11:8888/k8s/mysql:5.7
10.节点上删除镜像
#注意需要先删除标签镜像在删除源镜像
docker rmi 10.0.0.11:8888/k8s/mysql:5.7
docker rmi 10.0.0.11:8888/k8s/tomcat-app:v1
docker rmi mysql:5.7
docker rmi kubeguide/tomcat-app:v1
11.删除以前的demo项目 注意:主节点执行
[root@node1 ~]# kubectl delete -f tomcat-demo.yaml
deployment.apps "mysql" deleted
service "mysql" deleted
deployment.apps "myweb" deleted
service "myweb" deleted
12.修改demo项目的资源配置清单里的镜像地址
[root@node1 ~]# vim tomcat-demo.yaml #注意更改的位置
原来image: mysql:5.7 变更为: image: 10.0.0.11:8888/k8s/mysql:5.7
原来image: k8s/tomcat-app:v1 变更为: image: 10.0.0.11:8888/k8s/tomcat-app:v1
13.应用资源配置清单
[root@node1 ~]# kubectl create -f tomcat-demo.yaml
deployment.apps/mysql created
service/mysql created
deployment.apps/myweb created
service/myweb created
14.报错
#此时查看pod状态会发现镜像拉取失败了
[root@node1 ~]# kubectl get pod
NAME READY STATUS RESTARTS AGE
mysql-7d746b5577-wtxtm 0/1 ErrImagePull 0 15s
myweb-764df5ffdd-jvvmf 0/1 ImagePullBackOff 0 15s
myweb-764df5ffdd-rc9pc 0/1 ImagePullBackOff 0 15s
#查看pod创建的详细信息
[root@node1 ~]# kubectl describe pod mysql-7d746b5577-可以tab自己的数据
#关键报错信息:
Failed to pull image "10.0.0.11:8888/k8s/mysql:5.7": rpc error: code = Unknown desc = Error response from daemon: pull access denied for 10.0.0.11:8888/k8s/mysql, repository does not exist or may require 'docker login'
翻译:项目不出在或者需要登录
15.查看docker登陆的密码文件
[root@node1 ~]# docker login 10.0.0.11:8888
Authenticating with existing credentials...
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
#查看加密密码文件
[root@node1 ~]# cat /root/.docker/config.json
{
"auths": {
"10.0.0.11:8888": {
"auth": "YWRtaW46MTIzNDU2"
}
},
"HttpHeaders": {
"User-Agent": "Docker-Client/18.09.9 (linux)"
}
16.将docker密码文件解码成base64编码 解码:base64
[root@node1 ~]# cat /root/.docker/config.json|base64
ewoJImF1dGhzIjogewoJCSIxMC4wLjAuMTE6ODg4OCI6IHsKCQkJImF1dGgiOiAiWVdSdGFXNDZNVEl6TkRVMiIKCQl9Cgl9LAoJIkh0dHBIZWFkZXJzIjogewoJCSJVc2VyLUFnZW50IjogIkRvY2tlci1DbGllbnQvMTguMDkuOSAobGludXgpIgoJfQp9
#每一个人的都不一样
17.创建并应用docker登陆的Secret资源
#注意!!!
1.dockerconfigjson: xxx直接写base64的编码,不需要换行
2.base64编码是一整行,不是好几行
3.最后的type字段不能少
[root@node1 ~]# cat >harbor-secret.yaml<<EOF
apiVersion: v1
kind: Secret
metadata:
name: harbor-secret
data:
.dockerconfigjson: ewoJImF1dGhzIjogewoJCSIxMC4wLjAuMTE6ODg4OCI6IHsKCQkJImF1dGgiOiAiWVdSdGFXNDZNVEl6TkRVMiIKCQl9Cgl9LAoJIkh0dHBIZWFkZXJzIjogewoJCSJVc2VyLUFnZW50IjogIkRvY2tlci1DbGllbnQvMTguMDkuOSAobGludXgpIgoJfQp9
type: kubernetes.io/dockerconfigjson
EOF
#注意密码是一行
[root@node1 ~]# kubectl create -f harbor-secret.yaml
secret/harbor-secret created
[root@node1 ~]# kubectl get secrets
NAME TYPE DATA AGE
default-token-tslz6 kubernetes.io/service-account-token 3 23h
harbor-secret kubernetes.io/dockerconfigjson 1 46s
18.修改demo资源配置清单,添加拉取镜像的参数
查看命令帮助
kubectl explain deployment.spec.template.spec.imagePullSecrets
修改资源配置清单
修改文件
----------------------------
imagePullSecrets:
- name: harbor-secret
----------------------------
#注意:mysql和tomcat都需要增加
[root@node1 ~/demo]# cat tomcat-demo.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: mysql
spec:
replicas: 1
selector:
matchLabels:
app: mysql
template:
metadata:
labels:
app: mysql
spec:
containers:
- name: mysql
image: 10.0.0.11:8888/k8s/mysql:5.7
ports:
- containerPort: 3306
env:
- name: MYSQL_ROOT_PASSWORD
value: "123456"
imagePullSecrets:
- name: harbor-secret
---
apiVersion: v1
kind: Service
metadata:
name: mysql
spec:
ports:
- port: 3306
selector:
app: mysql
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: myweb
spec:
replicas: 2
selector:
matchLabels:
app: myweb
template:
metadata:
labels:
app: myweb
spec:
containers:
- name: myweb
image: 10.0.0.11:8888/k8s/tomcat-app:v1
ports:
- containerPort: 8080
env:
- name: MYSQL_SERVICE_HOST
value: 'mysql'
- name: MYSQL_SERVICE_PORT
value: '3306'
imagePullSecrets:
- name: harbor-secret
---
apiVersion: v1
kind: Service
metadata:
name: myweb
spec:
type: NodePort
ports:
- port: 8080
nodePort: 30001
selector:
app: myweb
19.应用资源配置清单并查看
1.删除资源配置清单
[root@node1 ~]# kubectl delete -f tomcat-demo.yaml
2.创建新的资源
[root@node1 ~]# kubectl create -f tomcat-demo.yaml
deployment.apps/mysql created
service/mysql created
deployment.apps/myweb created
service/myweb created
3.查询下载的资源
kubectl get pod -o wide
20.浏览器查看
报错总结:
#报错总结:
1.如果要删除的镜像正在被容器使用,那么你是删不了的
2.harbor卸载不干净,/data/harbor/目录下的数据也要删除
3.secret配置只写了一个dp,实际上有几个deployment就需要写几个
重做k8s使用harbor作为私有仓库
1.停止harbor正在运行的容器
2.删除harbor的容器
docker ps -a|grep "goharbor"|awk '{print "docker rm "$1}'
3.删除harbor的镜像
dockerimages|grep "goharbor"|awk '{print "docker rmi "$1":"$2}'
4.解压并修改harbor配置文件
hostname: 10.0.0.11
port: 8888
harbor_admin_password: 123456
data_volume: /data/harbor
5.执行安装并访问
./install.sh
http://10.0.0.11:8888
6.创建一个私有仓库k8s
7.配置docker信任仓库并重启--三台服务器都操作!!!
{
"registry-mirrors": ["https://ig2l319y.mirror.aliyuncs.com"],
"exec-opts": ["native.cgroupdriver=systemd"],
"insecure-registries" : ["http://10.0.0.11:8888"]
}
systemctl restart docker
注意!!!node1重启后harbor会失效,需要重启harbor
cd /opt/harbor
docker-compose stop
docker-compose start
8.docker登陆harbor
docker login 10.0.0.11:8888
9.将docker登陆凭证转化为k8s能识别的base64编码
[root@node1 ~]# cat /root/.docker/config.json|base64
ewoJImF1dGhzIjogewoJCSIxMC4wLjAuMTE6ODg4OCI6IHsKCQkJImF1dGgiOiAiWVdSdGFXNDZN
VEl6TkRVMiIKCQl9Cgl9LAoJIkh0dHBIZWFkZXJzIjogewoJCSJVc2VyLUFnZW50IjogIkRvY2tl
ci1DbGllbnQvMTguMDkuOSAobGludXgpIgoJfQp9
10.编写Secert资源配置清单
[root@node1 ~/demo]# cat harbor-secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: harbor-secret
data:
.dockerconfigjson: ewoJImF1dGhzIjogewoJCSIxMC4wLjAuMTE6ODg4OCI6IHsKCQkJImF1dGgiOiAiWVdSdGFXNDZNVEl6TkRVMiIKCQl9Cgl9LAoJIkh0dHBIZWFkZXJzIjogewoJCSJVc2VyLUFnZW50IjogIkRvY2tlci1DbGllbnQvMTguMDkuOSAobGludXgpIgoJfQp9
type: kubernetes.io/dockerconfigjson
11.应用Secret资源
kubectl delete -f harbor-secret.yaml
kubectl create -f harbor-secret.yaml
kubectl get secrets
12.修改镜像tag并上传到harbor
docker tag kubeguide/tomcat-app:v1 10.0.0.11:8888/k8s/tomcat-app:v1
docker tag mysql:5.7 10.0.0.11:8888/k8s/mysql:5.7
docker push 10.0.0.11:8888/k8s/tomcat-app:v1
docker push 10.0.0.11:8888/k8s/mysql:5.7
13.修改demo资源配置清单
####mysql
imagePullSecrets:
- name: harbor-secret
###tomcat
imagePullSecrets:
- name: harbor-secret
14.应用资源清单并查看
kubectl apply -f .
kubectl get pod