7--部署私有镜像仓库Harbor
目录
一、介绍
Harbor 是由 VMware 公司中国团队为企业用户设计的 Registry server 开源项目,包括了权限管理(RBAC)、LDAP、审计、管理界面、自我注册、HA 等企业必需的功能,同时针对中国用户的特点,设计镜像复制和中文支持等功能。作为一个企业级私有 Registry 服务器,Harbor 提供了更好的性能和安全。提升用户使用 Registry 构建和运行环境传输镜像的效率。Harbor 支持安装在多个 Registry 节点的镜像资源复制,镜像全部保存在私有 Registry 中, 确保数据和知识产权在公司内部网络中管控。另外,Harbor 也提供了高级的安全特性,诸如用户管理,访问控制和活动审计等。https://goharbor.io/
二、配置https证书
默认情况下,Harbor不附带证书。可以在没有安全性的情况下部署Harbor,以便您可以通过HTTP连接到它。但是,只有在没有连接到外部Internet的空白测试或开发环境中,才可以使用HTTP。在没有空隙的环境中使用HTTP会使您遭受中间人攻击。要配置HTTPS,必须创建SSL证书。您可以使用由受信任的第三方CA签名的证书,也可以使用自签名证书。本节介绍如何使用 OpenSSL创建CA,以及如何使用CA签署服务器证书和客户端证书。
1.生成证书颁发机构证书
在生产环境中,您应该从CA获得证书。在测试或开发环境中,您可以生成自己的CA。要生成CA证书,请运行以下命令。
1).生成证书私钥
[root@harbor ~]# mkdir /opt/cert
[root@harbor ~]# cd /opt/cert
[root@harbor /opt/cert]# openssl genrsa -out ca.key 4096
2).生成CA证书
openssl req -x509 -new -nodes -sha512 -days 3650 \
-subj "/C=CN/ST=ShangHai/L=ShangHai/O=Oldboy/OU=Linux/CN=192.168.15.110" \
-key ca.key \
-out ca.crt
[root@harbor /opt/cert]# openssl req -x509 -new -nodes -sha512 -days 3650 \
> -subj "/C=CN/ST=ShangHai/L=ShangHai/O=Oldboy/OU=Linux/CN=192.168.15.110" \
> -key ca.key \
> -out ca.crt
3).生成服务器证书
[root@harbor /opt/cert]# openssl genrsa -out 192.168.15.110.key 4096
4).生成证书签名请求
[root@harbor /opt/cert]# openssl req -sha512 -new \
-subj "/C=CN/ST=ShangHai/L=ShangHai/O=Oldboy/OU=Linux/CN=192.168.15.110" \
-key 192.168.15.110.key \
-out 192.168.15.110.csr
2、生成x509 va扩展文件
1)域名版
cat > v3.ext <<-EOF
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1=yourdomain.com
DNS.2=yourdomain
DNS.3=hostname
EOF
2). IP版
cat > v3.ext <<-EOF
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = IP:192.168.15.110
EOF
3、使用v3.ext文件生成证书
openssl x509 -req -sha512 -days 3650 \
-extfile v3.ext \
-CA ca.crt -CAkey ca.key -CAcreateserial \
-in 192.168.15.110.csr \
-out 192.168.15.110.crt
[root@harbor cert]# ll
total 28
-rw-r--r-- 1 root root 2053 Aug 25 11:37 192.168.15.110.crt
-rw-r--r-- 1 root root 1704 Aug 25 11:35 192.168.15.110.csr
-rw-r--r-- 1 root root 3247 Aug 25 11:34 192.168.15.110.key
-rw-r--r-- 1 root root 2029 Aug 25 11:33 ca.crt
-rw-r--r-- 1 root root 3243 Aug 25 11:32 ca.key
-rw-r--r-- 1 root root 17 Aug 25 11:37 ca.srl
-rw-r--r-- 1 root root 206 Aug 25 11:36 v3.ext
三、安装docker
#安装yum扩展命令
yum install yum-utils -y
#安装docker yum源
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
#读取生成yum缓存
yum makecache fast
#安装docker
yum -y install docker-ce
#启动docker
systemctl enable --now docker
四、提供证书给Harbor和Docker
生成ca.crt后,192.168.15.110.crt和192.168.15.110.key文件,必须将它们提供给Harbor和Docker,然后重启和启动他们。
#1.将服务器证书和密钥复制到Harbor主机上的certficates文件夹中
[root@harbor /opt/cert]# openssl x509 -inform PEM -in 192.168.15.110.crt -out 192.168.15.110.cert
[root@harbor /opt/cert]# mkdir -pv /etc/docker/certs.d/192.168.15.110/
[root@harbor /opt/cert]# cp 192.168.15.110.cert /etc/docker/certs.d/192.168.15.110/
[root@harbor /opt/cert]# cp 192.168.15.110.key /etc/docker/certs.d/192.168.15.110/
[root@harbor /opt/cert]# cp ca.crt /etc/docker/certs.d/192.168.15.110/
#2.如果nginx端口不是443和80(一般是不会改端口)
/etc/docker/certs.d/192.168.12.70:port
/etc/docker/certs.d/192.168.12.70:port
#3.复制Harbor证书
mkdir -p /data/cert
cp 192.168.15.110.crt /data/cert
cp 192.168.15.110.key /data/cert
cd /data/cert
#4.重启docker,加载证书
systemctl restart docker
五、安装Harbor
#1.下载
# 进入下载目录
cd /opt
wget https://github.com/goharbor/harbor/releases/download/v2.2.1/harbor-offline-installer-v2.2.1.tgz
#2.解压
[root@harbor /opt]# tar xf harbor-offline-installer-v2.2.1.tgz
#3.下载dockr-composrt
# 进入harbor目录
cd harbor/
[root@harbor harbor]# docker load < harbor.v2.2.1.tar.gz
# 下载docker-composrt
[root@harbor /opt/harbor]# curl -L "https://github.com/docker/compose/releases/download/1.24.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
#4.docker-compose添加可执行权限
chmod +x /usr/local/bin/docker-compose
#5.测试docker-composer
docker-compose -v
#6.修改harbor.yaml配置
# 复制成yml。并改配置
[root@harbor /opt/harbor]# cp harbor.yml.tmpl harbor.yml
...
hostname: 192.168.15.110
...
certificate: /data/cert/192.168.15.110.crt
private_key: /data/cert/192.168.15.110.key
...
#7.生成配置文件
[root@harbor /opt/harbor]# ./prepare
#8.清空生成配置的残留
[root@harbor /opt/harbor]# docker-compose down
#9.安装并启动
[root@harbor /opt/harbor]# ./install.sh
#10.访问
URL:http://192.168.15.110
用户名:admin
密码:Harbor12345(harbor.yml 里面的默认密码)
#11.docke测试登录
[root@harbor /opt/harbor]# docker login 192.168.15.110 --username=admin --password=Harbor12345
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
#12.分发证书到k8s集群所有节点
[root@harbor ~]# cd /opt/cert/
[root@harbor cert]# for i in 192.168.15.13 192.168.15.12;
do
ssh root@$i "mkdir -pv /etc/docker/c erts.d/192.168.15.110/"
scp 192.168.15.110.cert root@$i:/etc/docker/certs.d/192.168.15.110/
scp 192.168.15.110.key root@$i:/etc/docker/certs.d/192.168.15.110/
scp ca.crt root@$i:/etc/docker/certs.d/192.168.15.110/
done
#13.node节点重启docker
[root@k8s-n-01 ~]# systemctl restart docker
#node节点登录仓库成功即可
[root@k8s-n-01 ~]# docker login 192.168.15.110
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
六、Jenkins使用kubernetes
1.创建admin-csr.json(kubernetes)
[root@k8s-master-01 ~]# mkdir /opt/cert && cd /opt/cert
[root@k8s-master-01 /opt/cert]# cat > admin-csr.json << EOF
> {
> "CN":"admin",
> "key":{
> "algo":"rsa",
> "size":2048
> },
> "names":[
> {
> "C":"CN",
> "L":"BeiJing",
> "ST":"BeiJing",
> "O":"system:masters",
> "OU":"System"
> }
> ]
> }
> EOF
2.安装证书生成工具
#1.下载工具
wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
#2.添加执行权限
[root@k8s-master-01 /opt/cert]# chmod +x cfssljson_linux-amd64
[root@k8s-master-01 /opt/cert]# chmod +x cfssl_linux-amd64
#3.移动到/usr/local/bin
[root@k8s-master-01 /opt/cert]# mv cfssljson_linux-amd64 cfssljson
[root@k8s-master-01 /opt/cert]# mv cfssl_linux-amd64 cfssl
[root@k8s-master-01 /opt/cert]# mv cfssljson cfssl /usr/local/bin
#4.创建证书私钥
[root@k8s-master-01 /opt/cert]# cfssl gencert -ca=/etc/kubernetes/pki/ca.crt -ca-key=/etc/kubernetes/pki/ca.key --profile=kubernetes admin-csr.json | cfssljson -bare admin
#5.配置证书(kubernetes)
[root@k8s-master-01 /opt/cert]# openssl pkcs12 -export -out ./jenkins-admin.pfx -inkey ./admin-key.pem -in ./admin.pem -passout pass:123456
#6.配置jenkins连接k8s(jenkins)
下载证书
[root@k8s-master-01 ~]# cd /opt/cert/
[root@k8s-master-01 /opt/cert]# ll
...
-rw-r--r-- 1 root root 2517 Apr 20 19:59 jenkins-admin.pfx
...
[root@k8s-master-01 /opt/cert]# sz jenkins-admin.pfx
#7.连接k8s测试
[root@k8s-master-01 ~]# kubectl create secret generic kubeconfig --from-file=.kube/config
secret/kubeconfig created