linux运维、架构之路-企业级镜像仓库Harbor主从复制
一、harbor介绍
Harbor是由VMware公司开源的企业级的Docker Registry管理项目，它包括权限管理(RBAC)、LDAP、日志审核、管理界面、自我注册、镜像复制和中文支持等功能。
Harbor主要组件包括proxy，它是一个nginx前端代理，主要是分发前端页面ui访问和镜像上传和下载流量，Ui提供了一个Web管理界面，当然还包括了一个前端页面和后端API，底层使用mysql数据库；Registry是镜像仓库，负责存储镜像文件，当镜像上传完毕后通过ui创建Registry。 当然registry的token认证也是通过ui组件完成；adminserver是系统的配置管理中心附带检查存储用量，ui和Jobserver启动时候会需要加载adminserver的配置;jobservice是负载镜像复制工作的，他和registry通信，从一个registry pull镜像然后push到另一个registry并记录job_log,log是日志汇总组件，通过docker的log-driver把日志汇总到一起。
Harbor 的每个组件都是以 Docker 容器的形式构建的，我们使用 Docker Compose 来对它进行部署。
二、Harbor组件
- proxy：由 Nginx 服务器构成的反向代理。
- registry：由 Docker 官方的开源 registry 镜像构成的容器实例。
- ui：即架构中的 core services, 构成此容器的代码是 Harbor 项目的主体。
- mysql：由官方 MySql 镜像构成的数据库容器。
- log: 运行着 rsyslogd 的容器，通过 log-driver 的形式收集其他容器的日志。
三、Harbor部署
1、服务器规划
|
系统 |
IP地址 |
主机名 |
CPU |
内存 |
|
CentOS 7.5 |
192.168.56.10 |
Harbor01 |
2C |
2G |
|
CentOS 7.5 |
192.168.56.11 |
harbor02 |
2C |
2G |
2、关闭防火墙、SElinux等
setenforce 0 systemctl stop firewalld systemctl disable firewalld sed -i 's/enforcing/disabled/g' /etc/sysconfig/selinux
3、Docker部署
下载地址：https://download.docker.com/linux/centos/7/x86_64/stable/Packages/
①安装Docker18.09.6
wget https://download.docker.com/linux/centos/7/x86_64/stable/Packages/containerd.io-1.2.5-3.1.el7.x86_64.rpm wget https://download.docker.com/linux/centos/7/x86_64/stable/Packages/docker-ce-18.09.6-3.el7.x86_64.rpm wget https://download.docker.com/linux/centos/7/x86_64/stable/Packages/docker-ce-cli-18.09.6-3.el7.x86_64.rpm wget https://download.docker.com/linux/centos/7/x86_64/stable/Packages/docker-ce-selinux-17.03.3.ce-1.el7.noarch.rpm
yum -y install docker-ce-selinux-17.03.3.ce-1.el7.noarch.rpm yum -y install containerd.io-1.2.5-3.1.el7.x86_64.rpm yum -y install docker-ce-cli-18.09.6-3.el7.x86_64.rpm yum -y install docker-ce-18.09.6-3.el7.x86_64.rpm
②镜像加速
mkdir /etc/docker -p && mkdir /app/docker_data -p cat > /etc/docker/daemon.json <<EOF { "registry-mirrors": ["https://docker.mirrors.ustc.edu.cn","http://hub-mirror.c.163.com","https://dockerhub.mirrors.nwafu.edu.cn"], "insecure-registries": ["http://harbor.clevercode.net","http://harbor01.clevercode.net","http://harbor02.clevercode.net","http://harbor03.clevercode.net"], "max-concurrent-downloads": 10, "log-driver": "json-file", "log-level": "warn", "log-opts": { "max-size": "10m", "max-file": "3" }, "data-root": "/app/docker_data" } EOF
③启动Docker服务
systemctl daemon-reload && systemctl restart docker && docker info Docker Root Dir: /app/docker_data Debug Mode (client): false Debug Mode (server): false Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: harbor02.clevercode.net harbor03.clevercode.net harbor.clevercode.net harbor01.clevercode.net 127.0.0.0/8 Registry Mirrors: https://docker.mirrors.ustc.edu.cn/ http://hub-mirror.c.163.com/ https://dockerhub.mirrors.nwafu.edu.cn/ Live Restore Enabled: false Product License: Community Engine
4、docker-compose部署
curl -L https://github.com/docker/compose/releases/download/1.21.2/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose chmod +x /usr/local/bin/docker-compose
注：服务器无法访问外网的情况，可以保存此包上传到/usr/local/bin目录
5、Harbor主从复制部署
①下载地址
地址：https://github.com/goharbor/harbor/releases
如果服务器不能出访外网，下载如下离线包
wget https://github.com/goharbor/harbor/releases/download/v1.10.3/harbor-online-installer-v1.10.3.tgz tar xf harbor-online-installer-v1.10.3.tgz -C /app/ && cd /app/harbor/ [root@harbor01 harbor]# head -20 harbor.yml # Configuration file of Harbor # The IP address or hostname to access admin UI and registry service. # DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients. hostname: 192.168.56.10 # http related config http: # port for http, default is 80. If https enabled, this port will redirect to https port port: 8080 # https related config #https: # https port for harbor, default is 443 # port: 443 # The path of cert and key files for nginx # certificate: /your/certificate/path # private_key: /your/private/key/path # Uncomment external_url if you want to enable external proxy
为了方便测试，注释掉https认证，不然会报错
ERROR:root:Error: The protocol is https but attribute ssl_cert is not set
②执行安装：
./install.sh #执行安装操作
# ./prepare #预处理
Creating harbor-log ... done Creating redis ... done Creating harbor-db ... done Creating registryctl ... done Creating harbor-portal ... done Creating registry ... done Creating harbor-core ... done Creating nginx ... done Creating harbor-jobservice ... done ✔ ----Harbor has been installed and started successfully.----
#Harbor启动停止命令
docker-compose up -d #后台启动，如果容器不存在根据镜像自动创建
docker-compose down -v #使用此命令停止容器并删除容器更改路径才生效
③查看
[root@harbor01 harbor]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES a382bb9a7052 goharbor/harbor-jobservice:v1.10.3 "/harbor/harbor_jobs…" 18 minutes ago Up 18 minutes (healthy) harbor-jobservice 809e4a300d2f goharbor/nginx-photon:v1.10.3 "nginx -g 'daemon of…" 18 minutes ago Up 18 minutes (healthy) 0.0.0.0:8080->8080/tcp nginx ccd35f5c5a71 goharbor/harbor-core:v1.10.3 "/harbor/harbor_core" 18 minutes ago Up 18 minutes (healthy) harbor-core 081ab4e8ae5a goharbor/registry-photon:v1.10.3 "/home/harbor/entryp…" 19 minutes ago Up 18 minutes (healthy) 5000/tcp registry 4e04674837e4 goharbor/harbor-portal:v1.10.3 "nginx -g 'daemon of…" 19 minutes ago Up 18 minutes (healthy) 8080/tcp harbor-portal aca38609460f goharbor/harbor-registryctl:v1.10.3 "/home/harbor/start.…" 19 minutes ago Up 18 minutes (healthy) registryctl 1e79924827bf goharbor/harbor-db:v1.10.3 "/docker-entrypoint.…" 19 minutes ago Up 18 minutes (healthy) 5432/tcp harbor-db 49b5861eeae4 goharbor/redis-photon:v1.10.3 "redis-server /etc/r…" 19 minutes ago Up 18 minutes (healthy) 6379/tcp redis 16fc65cb671c goharbor/harbor-log:v1.10.3 "/bin/sh -c /usr/loc…" 19 minutes ago Up 19 minutes (healthy) 127.0.0.1:1514->10514/tcp harbor-log
④登录验证
http://192.168.56.10:8080/ http://192.168.56.11:8080/ 默认用户密码：admin/Harbor12345
⑤配置Docker向harbor中推送镜像
[root@harbor01 harbor]# head /etc/docker/daemon.json { "registry-mirrors": ["https://docker.mirrors.ustc.edu.cn","http://hub-mirror.c.163.com","https://dockerhub.mirrors.nwafu.edu.cn"], "insecure-registries": ["http://harbor.clevercode.net","http://harbor01.clevercode.net","http://harbor02.clevercode.net","http://harbor03.clevercode.net"], "insecure-registries": ["192.168.56.10:8080"], "max-concurrent-downloads": 10, "log-driver": "json-file", "log-level": "warn", "log-opts": { "max-size": "10m",
重启Docker服务：
systemctl daemon-reload && systemctl restart docker
登录Harbor仓库：
[root@harbor01 harbor]# docker login -u admin -p Harbor12345 192.168.56.10:8080 WARNING! Using --password via the CLI is insecure. Use --password-stdin. WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded
Harbor新建项目：
推送镜像到Harbor仓库：
[root@harbor01 harbor]# docker tag busybox:latest 192.168.56.10:8080/demon/busybox:v1 [root@harbor01 harbor]# [root@harbor01 harbor]# docker push 192.168.56.10:8080/demon/busybox:v1 The push refers to repository [192.168.56.10:8080/demon/busybox] 0064d0478d00: Pushed v1: digest: sha256:0415f56ccc05526f2af5a7ae8654baec97d4a614f24736e8eef41a4591f08019 size: 527
6、Harbor主从复制配置
- harbor01: 192.168.56.10 (主)
- harbor02: 192.168.56.11 (从)
①Harbor01上面配置
②创建复制规则
③推送镜像触发镜像复制
[root@harbor01 harbor]# docker tag nginx:latest 192.168.56.10:8080/demon/nginx:v1 [root@harbor01 harbor]# docker push 192.168.56.10:8080/demon/nginx:v1 The push refers to repository [192.168.56.10:8080/demon/nginx] 85fcec7ef3ef: Pushed 3e5288f7a70f: Pushed 56bc37de0858: Pushed 1c91bf69a08b: Pushed cb42413394c4: Pushed v1: digest: sha256:0b159cd1ee1203dad901967ac55eee18c24da84ba3be384690304be93538bea8 size: 1362
④登录Harbor02验证
发现刚推送的nginx镜像已经同步到从节点
成功最有效的方法就是向有经验的人学习！
