深入学习Kubernetes(四)：部署harbor、kubernets怎么使用habor
Habor是由VMWare中国团队开源的容器镜像仓库。事实上，Habor是在Docker Registry上进行了相应的企业级扩展，从而获 得了更加广泛的应用，这些新的企业级特性包括：管理用户界面，基于角色的访问控制 ，水平扩展，同步，AD/LDAP集成以及 审计日志等。
一. 部署harbor
1. 下载最新的docker-compose二进制文件
# curl -L https://github.com/docker/compose/releases/download/1.22.0-rc1/docker-compose-`uname -s`-`uname -m` -o /usr/bin/docker-compose
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 617 0 617 0 0 415 0 --:--:-- 0:00:01 --:--:-- 415
100 11.2M 100 11.2M 0 0 176k 0 0:01:05 0:01:05 --:--:-- 185k
#chmod 755 /usr/bin/docker-compose
2. 下载harbor离线安装包
harbor官方github地址 https://github.com/goharbor/harbor/releases # wget https://storage.googleapis.com/harbor-releases/release-1.6.0/harbor-offline-installer-v1.6.0-rc3.tgz # tar zxvf harbor-offline-installer-v1.6.0-rc3.tgz
3. 准备ca证书（建议大家购买一个便宜的域名，并去申请一个免费的ssl证书）
[root@node1 ~]# ls
anaconda-ks.cfg cert harbor harbor-offline-installer-v1.6.0.tgz
[root@node1 ~]# mkdir -p /data/cert
//创建证书目录
# mv cert/server.* /data/cert
//把证书文件移动到/data/cert目录下
-
详见另一博文ssl证书申请
4. 安装harbor
4.1编辑配置文件
# cd harbor 编辑配置文件 # vim harbor.cfg //定义hostname hostname = harbor.wangjianhua.xyz //定义ui_url_protocol为https ui_url_protocol = https //定义ssl_cert/ssl_key ssl_cert = /data/cert/server.crt ssl_cert_key = /data/cert/server.key //定义harbor_admin_password harbor_admin_password = Harbor12345
4.2 安装
[root@node1 ~]# cd harbor/ [root@node1 harbor]# sh install.sh [Step 0]: checking installation environment ... Note: docker version: 18.06.1 Note: docker-compose version: 1.22.0 [Step 1]: loading Harbor images ... Loaded image: goharbor/harbor-log:v1.6.0 Loaded image: goharbor/harbor-db:v1.6.0 Loaded image: goharbor/nginx-photon:v1.6.0 Loaded image: goharbor/notary-server-photon:v0.5.1-v1.6.0 Loaded image: goharbor/chartmuseum-photon:v0.7.1-v1.6.0 Loaded image: goharbor/harbor-ui:v1.6.0 Loaded image: goharbor/harbor-jobservice:v1.6.0 Loaded image: goharbor/redis-photon:v1.6.0 Loaded image: goharbor/registry-photon:v2.6.2-v1.6.0 Loaded image: goharbor/notary-signer-photon:v0.5.1-v1.6.0 Loaded image: goharbor/clair-photon:v2.0.5-v1.6.0 Loaded image: goharbor/harbor-migrator:v1.6.0 Loaded image: goharbor/harbor-adminserver:v1.6.0 [Step 2]: preparing environment ... Clearing the configuration file: ./common/config/adminserver/env Clearing the configuration file: ./common/config/ui/env Clearing the configuration file: ./common/config/ui/app.conf Clearing the configuration file: ./common/config/ui/private_key.pem Clearing the configuration file: ./common/config/db/env Clearing the configuration file: ./common/config/jobservice/env Clearing the configuration file: ./common/config/jobservice/config.yml Clearing the configuration file: ./common/config/registry/config.yml Clearing the configuration file: ./common/config/registry/root.crt Clearing the configuration file: ./common/config/registryctl/env Clearing the configuration file: ./common/config/registryctl/config.yml Clearing the configuration file: ./common/config/nginx/cert/server.crt Clearing the configuration file: ./common/config/nginx/cert/server.key Clearing the configuration file: ./common/config/nginx/nginx.conf Clearing the configuration file: ./common/config/log/logrotate.conf loaded secret from file: /data/secretkey Generated configuration file: ./common/config/nginx/nginx.conf Generated configuration file: ./common/config/adminserver/env Generated configuration file: ./common/config/ui/env Generated configuration file: ./common/config/registry/config.yml Generated configuration file: ./common/config/db/env Generated configuration file: ./common/config/jobservice/env Generated configuration file: ./common/config/jobservice/config.yml Generated configuration file: ./common/config/log/logrotate.conf Generated configuration file: ./common/config/registryctl/env Generated configuration file: ./common/config/ui/app.conf Generated certificate, key file: ./common/config/ui/private_key.pem, cert file: ./common/config/registry/root.crt The configuration files are ready, please use docker-compose to start the service. [Step 3]: checking existing instance of Harbor ... Note: stopping existing Harbor instance ... Stopping redis ... done Stopping harbor-adminserver ... done Stopping harbor-log ... done Removing registry ... done Removing redis ... done Removing harbor-db ... done Removing harbor-adminserver ... done Removing harbor-log ... done Removing network harbor_harbor [Step 4]: starting Harbor ... Creating network "harbor_harbor" with the default driver Creating harbor-log ... done Creating harbor-db ... done Creating harbor-adminserver ... done Creating redis ... done Creating registry ... done Creating harbor-ui ... done Creating harbor-jobservice ... done Creating nginx ... done ✔ ----Harbor has been installed and started successfully.---- Now you should be able to visit the admin portal at https://harbor.yuankeedu.com. For more details, please visit https://github.com/goharbor/harbor .
4.3查看下载的镜像：
[root@node1 harbor]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 9efb51e18d6f goharbor/nginx-photon:v1.6.0 "nginx -g 'daemon of…" 19 minutes ago Up 19 minutes (healthy) 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:4443->4443/tcp nginx 74374990c433 goharbor/harbor-jobservice:v1.6.0 "/harbor/start.sh" 19 minutes ago Up 19 minutes harbor-jobservice 71aca3a0a108 goharbor/harbor-ui:v1.6.0 "/harbor/start.sh" 19 minutes ago Up 19 minutes (healthy) harbor-ui d949ec62a255 goharbor/harbor-db:v1.6.0 "/entrypoint.sh post…" 19 minutes ago Up 19 minutes (healthy) 5432/tcp harbor-db 53318d9d8384 goharbor/registry-photon:v2.6.2-v1.6.0 "/entrypoint.sh /etc…" 19 minutes ago Up 19 minutes (healthy) 5000/tcp registry 1a25f0b81e6b goharbor/redis-photon:v1.6.0 "docker-entrypoint.s…" 19 minutes ago Up 19 minutes 6379/tcp redis ec88e470f024 goharbor/harbor-adminserver:v1.6.0 "/harbor/start.sh" 19 minutes ago Up 19 minutes (healthy) harbor-adminserver 80c8594b4ac9 goharbor/harbor-log:v1.6.0 "/bin/sh -c /usr/loc…" 19 minutes ago Up 19 minutes (healthy) 127.0.0.1:1514->10514/tcp harbor-log 534bcf6cbbfa f0fad859c909 "/opt/bin/flanneld -…" 21 minutes ago Up 21 minutes k8s_kube-flannel_kube-flannel-ds-mlt9l_kube-system_0ecc0543-17a5-11e9-acec-000c29e57c32_2 ee50d44c5e3e mirrorgooglecontainers/pause-amd64:3.1 "/pause" 21 minutes ago Up 21 minutes k8s_POD_kube-flannel-ds-mlt9l_kube-system_0ecc0543-17a5-11e9-acec-000c29e57c32_2 [root@node1 harbor]# docker-compose down //停掉启动的所有容器 [root@node1 harbor]# docker-compose up -d //开启所有的容器,-d丢到后台启动
5.访问
5.1 添加域名解析
打开windwos下C:\Windows\System32\drivers\etc\hosts
# Copyright (c) 1993-2009 Microsoft Corp.
#
# This is a sample HOSTS file used by Microsoft TCP/IP for Windows.
#
# This file contains the mappings of IP addresses to host names. Each
# entry should be kept on an individual line. The IP address should
# be placed in the first column followed by the corresponding host name.
# The IP address and the host name should be separated by at least one
# space.
#
# Additionally, comments (such as these) may be inserted on individual
# lines or following the machine name denoted by a '#' symbol.
#
# For example:
#
# 102.54.94.97 rhino.acme.com # source server
# 38.25.63.10 x.acme.com # x client host
# localhost name resolution is handled within DNS itself.
# 127.0.0.1 localhost
# ::1 localhost
192.168.0.111 harbor.wangjianhua.xyz
#添加域名和IP
5.2 登陆
打开URL:https://harbor.wangjianhua.xyz/harbor/sign-in 用户名：admin 密码：Harbor12345
6. 创建项目
点击新建项目—输入项目名称
- 选择公开，所有人都能看到。
7. 拉取公共镜像
[root@node1 harbor]#docker pull tomcat [root@node1 harbor]# docker tag tomcat harbor.wangjianhua.xyz/wjh/tomcat:latest //打标签
8. 把tomcat镜像推送到harbor
[root@node1 harbor]# docker login https://harbor.wangjianhua.xyz Username: admin Password: WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded [root@node1 harbor]# docker push harbor.wangjianhua.xyz/wjh/tomcat:latest The push refers to repository [harbor.wangjianhua.xyz/wjh/tomcat] 10acf9008214: Pushed f725cace5e8c: Pushed 70333c64c1a2: Pushed a8586e8d29fb: Pushed 2d63501da7e5: Pushed e6ae888dd260: Pushed 80917357f055: Pushed bfc5dbc4d7a7: Pushed b53b57a50746: Pushed d2518892581f: Pushed c581f4ede92d: Pushed latest: digest: sha256:af0dc8139f8353ff2f10f8fe9fdf54a9d844f4c337b35dd40b6c0336d919e3f6 size: 2626
在web上查看：
问题：登陆时报509
[root@node1 harbor]# docker login https://harbor.wangjianhua.xyz Username: admin Password: Error response from daemon: Get https://harbor.wangjianhua.xyz/v2/: x509: certificate signed by unknown authority
解决方法：
[root@node1 harbor]# mkdir -p /etc/docker/certs.d/harbor.wangjianhua.xyz [root@node1 harbor]# vim /etc/docker/certs.d/harbor.wangjianhua.xyz/ca.crt -----BEGIN CERTIFICATE----- MIIErjCCA5agAwIBAgIQBYAmfwbylVM0jhwYWl7uLjANBgkqhkiG9w0BAQsFADBh MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3 d3cuZGlnaWNlcnQuY29tMSAwHgYDVQQDExdEaWdpQ2VydCBHbG9iYWwgUm9vdCBD QTAeFw0xNzEyMDgxMjI4MjZaFw0yNzEyMDgxMjI4MjZaMHIxCzAJBgNVBAYTAkNO MSUwIwYDVQQKExxUcnVzdEFzaWEgVGVjaG5vbG9naWVzLCBJbmMuMR0wGwYDVQQL ExREb21haW4gVmFsaWRhdGVkIFNTTDEdMBsGA1UEAxMUVHJ1c3RBc2lhIFRMUyBS U0EgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCgWa9X+ph+wAm8 Yh1Fk1MjKbQ5QwBOOKVaZR/OfCh+F6f93u7vZHGcUU/lvVGgUQnbzJhR1UV2epJa e+m7cxnXIKdD0/VS9btAgwJszGFvwoqXeaCqFoP71wPmXjjUwLT70+qvX4hdyYfO JcjeTz5QKtg8zQwxaK9x4JT9CoOmoVdVhEBAiD3DwR5fFgOHDwwGxdJWVBvktnoA zjdTLXDdbSVC5jZ0u8oq9BiTDv7jAlsB5F8aZgvSZDOQeFrwaOTbKWSEInEhnchK ZTD1dz6aBlk1xGEI5PZWAnVAba/ofH33ktymaTDsE6xRDnW97pDkimCRak6CEbfe 3dXw6OV5AgMBAAGjggFPMIIBSzAdBgNVHQ4EFgQUf9OZ86BHDjEAVlYijrfMnt3K AYowHwYDVR0jBBgwFoAUA95QNVbRTLtm8KPiGxvDl7I90VUwDgYDVR0PAQH/BAQD AgGGMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjASBgNVHRMBAf8ECDAG AQH/AgEAMDQGCCsGAQUFBwEBBCgwJjAkBggrBgEFBQcwAYYYaHR0cDovL29jc3Au ZGlnaWNlcnQuY29tMEIGA1UdHwQ7MDkwN6A1oDOGMWh0dHA6Ly9jcmwzLmRpZ2lj ZXJ0LmNvbS9EaWdpQ2VydEdsb2JhbFJvb3RDQS5jcmwwTAYDVR0gBEUwQzA3Bglg hkgBhv1sAQIwKjAoBggrBgEFBQcCARYcaHR0cHM6Ly93d3cuZGlnaWNlcnQuY29t L0NQUzAIBgZngQwBAgEwDQYJKoZIhvcNAQELBQADggEBAK3dVOj5dlv4MzK2i233 lDYvyJ3slFY2X2HKTYGte8nbK6i5/fsDImMYihAkp6VaNY/en8WZ5qcrQPVLuJrJ DSXT04NnMeZOQDUoj/NHAmdfCBB/h1bZ5OGK6Sf1h5Yx/5wR4f3TUoPgGlnU7EuP ISLNdMRiDrXntcImDAiRvkh5GJuH4YCVE6XEntqaNIgGkRwxKSgnU3Id3iuFbW9F UQ9Qqtb1GX91AJ7i4153TikGgYCdwYkBURD8gSVe8OAco6IfZOYt/TEwii1Ivi1C qnuUlWpsF1LdQNIdfbW3TSe0BhQa7ifbVIfvPWHYOu3rkg1ZeMo6XRU9B4n5VyJY RmE= -----END CERTIFICATE----- "/etc/docker/certs.d/harbor.wangjianhua.xyz/ca.crt" [新] 28L, 1684C 已写入
//注意，这个只针对harbor.wangjianhua.xyz域名的，这个其实就是免费的ca证书压缩包里的ca文件内容 [root@node1 harbor]# !system systemctl restart docker
//注意kubernets的所有节点都需要做这种认证操作，否则会报509错误
二、kubernetes中使用harbor
以下操作在master上执行：
2.1创建secret
[root@marster1 k8s]# kubectl create secret docker-registry my-secret --docker-server=harbor.wangjianhua.xyz --docker-username=admin --docker-password=Harbor12345 secret/my-secret created
创建完成后，可以用以下命令查看：
[root@marster1 k8s]# kubectl get secret NAME TYPE DATA AGE default-token-6g47n kubernetes.io/service-account-token 3 2d my-secret kubernetes.io/dockerconfigjson 1 23m
- 创建secret的作用：是把harbor 的用户名密码放到这个媒介中，所有的节点就可以访问了
2.2 定义一个pod
[root@marster1 k8s]# docker pull httpd [root@marster1 k8s]# docker tag httpd harbor.wangjianhua.xyz/wjh/httpd:latest [root@marster1 k8s]# docker login https://harbor.wangjianhua.xyz [root@marster1 ~]# docker push harbor.wangjianhua.xyz/wjh/httpd:latest //首先，需要在harbo私有仓库里推送一个httpd的镜像，地址为harbor.wangjianhua.xyz/wjh/httpd:latest
2.3 定义httpd.yaml文件
[root@marster1 ~]# vim httpd.yaml apiVersion: v1 kind: Pod metadata: name: httpd-pod spec: containers: - image: harbor.wangjianhua.xyz/wjh/httpd:latest name: httpd-pod imagePullSecrets: - name: my-secret [root@marster1 ~]# kubectl create -f httpd.yaml pod/httpd-pod created [root@marster1 ~]# kubectl create -f httpd.yaml pod/httpd-pod created [root@marster1 ~]# kubectl get pod NAME READY STATUS RESTARTS AGE httpd-pod 1/1 Running 0 34s nginx-6f858d4d45-4s55d 1/1 Running 5 2h
