Jenkins之docker in docker用https登录harbor
以下是官网的基础环境
先创建network
docker network create jenkins
运行容器jenkins-docker，网络别名为docker。
docker run \
--name jenkins-docker \
--rm \
--detach \
--privileged \
--network jenkins \
--network-alias docker \
--env DOCKER_TLS_CERTDIR=/certs \
--volume jenkins-docker-certs:/certs/client \
--volume jenkins-data:/var/jenkins_home \
--publish 2376:2376 \
--publish 3000:3000 --publish 5000:5000 \
docker:dind \
--storage-driver overlay2
vi Dockerfile
FROM jenkins/jenkins:2.387.1
USER root
RUN apt-get update && apt-get install -y lsb-release
RUN curl -fsSLo /usr/share/keyrings/docker-archive-keyring.asc \
https://download.docker.com/linux/debian/gpg
RUN echo "deb [arch=$(dpkg --print-architecture) \
signed-by=/usr/share/keyrings/docker-archive-keyring.asc] \
https://download.docker.com/linux/debian \
$(lsb_release -cs) stable" > /etc/apt/sources.list.d/docker.list
RUN apt-get update && apt-get install -y docker-ce-cli
USER jenkins
RUN jenkins-plugin-cli --plugins "blueocean docker-workflow"
构建
docker build -t myjenkins-blueocean:2.387.1-1 .
Run your own myjenkins-blueocean:2.387.1-1 image as a container in Docker
docker run \
--name jenkins-blueocean \
--detach \
--network jenkins \
--env DOCKER_HOST=tcp://docker:2376 \
--env DOCKER_CERT_PATH=/certs/client \
--env DOCKER_TLS_VERIFY=1 \
--publish 8080:8080 \
--publish 50000:50000 \
--volume jenkins-data:/var/jenkins_home \
--volume jenkins-docker-certs:/certs/client:ro \
--volume "$HOME":/home \
--restart=on-failure \
--env JAVA_OPTS="-Dhudson.plugins.git.GitSCM.ALLOW_LOCAL_CHECKOUT=true" \
myjenkins-blueocean:2.387.1-1
写一个自己的pipleline
node {
stage('Clone') {
echo "1.Clone Stage"
git url: "https://github.com/baidjay/jenkins-demo.git"
script {
build_tag = sh(returnStdout: true, script: 'git rev-parse --short HEAD').trim()
}
echo "${build_tag}"
}
stage('Test') {
echo "2.Test Stage"
}
stage('Build') {
echo "3.Build Docker Image Stage"
sh "docker build -t 192.168.1.143:81/jenkins/jenkins-demo:${build_tag} ."
}
stage('Push') {
echo "4.Push Docker Image Stage"
sh "docker login --username=admin 192.168.1.143:81 -p Harbor12345"
sh "docker push https://allinone.st.com/jenkins/jenkins-demo:${build_tag}"
}
stage('YAML') {
echo "5. Change YAML File Stage"
def userInput = input(
id: 'userInput',
message: 'Choose a deploy environment',
parameters: [
[
$class: 'ChoiceParameterDefinition',
choices: "Dev\nQA\nProd",
name: 'Env'
]
]
)
echo "This is a deploy step to ${userInput}"
sh "sed -i 's/<BUILD_TAG>/${build_tag}/' k8s.yaml"
sh "sed -i 's/<BRANCH_NAME>/${userInput}/' k8s.yaml"
sh "sed -i 's#registry.cn-hangzhou.aliyuncs.com/ik9s#harbor.st.com/jenkins#' k8s.yaml"
sh "cat k8s.yaml"
}
stage('Deploy') {
echo "6. Deploy Stage"
withKubeConfig([credentialsId: '2bc4be96-168a-4033-ba16-51eb0e21xxx', serverUrl: 'https://192.168.1.151:6443']) {
sh 'kubectl apply -f k8s.yaml'
}
}
}
运行出错
使用harbor http方式连接，提示要用https连接
官网教程: Configure HTTPS Access to Harbor
https://goharbor.io/docs/1.10/install-config/configure-https/
https://goharbor.io/docs/2.6.0/install-config/configure-https/
Generate a Certificate Authority Certificate
openssl genrsa -out ca.key 4096
openssl req -x509 -new -nodes -sha512 -days 3650 \
-subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=st.com" \
-key ca.key \
-out ca.crt
Generate a Server Certificate
The certificate usually contains a .crt file and a .key file
openssl genrsa -out st.com.key 4096
openssl req -sha512 -new \
-subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=st.com" \
-key st.com.key \
-out st.com.csr
cat > v3.ext <<-EOF
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1=st.com
DNS.2=st
DNS.3=harbor.st.com
DNS.4=harbor
EOF
openssl x509 -req -sha512 -days 3650 \
-extfile v3.ext \
-CA ca.crt -CAkey ca.key -CAcreateserial \
-in st.com.csr \
-out st.com.crt
Provide the Certificates to Harbor and Docker
After generating the ca.crt, yourdomain.com.crt, and yourdomain.com.key files, you must provide them to Harbor and to Docker, and reconfigure Harbor to use them.
Copy the server certificate and key into the certficates folder on your Harbor host.
cp st.com.crt /data/cert/
cp st.com.key /data/cert/
openssl x509 -inform PEM -in st.com.crt -out st.com.cert
mkdir /etc/docker/certs.d/st.com
cp st.com.cert /etc/docker/certs.d/st.com/
cp st.com.key /etc/docker/certs.d/st.com/
cp ca.crt /etc/docker/certs.d/st.com/
如果不是443端口，文件夹命名为yourdomain.com:port
If you mapped the default nginx port 443 to a different port, create the folder /etc/docker/certs.d/yourdomain.com:port, or /etc/docker/certs.d/harbor_IP:port.
systemctl restart docker
操作系统级别信任证书
cp ca.crt /etc/pki/ca-trust/source/anchors/st.com.crt
update-ca-trust extract
systemctl restart docker
(harbor要login一次)
注意：是ca.crt，不是 yourdomain.com..crt （官网是先提前操作cat intermediate-certificate.pem >> yourdomain.com.crt）
https://goharbor.io/docs/2.6.0/install-config/troubleshoot-installation/#https
harbor配置文件启用https
vim harbor.yml
hostname: harbor.st.com
http:
port: 80
https
port: 4443
#存放证书的目录
certificate: /data/cert/st.com.crt
private_key: /data/cert/st.com.key
harbor_admin_password: Harbor12345
data_volume: /data/harbor
log
location: /data/harbor/logs
docker-compose down
重新用配置文件启动，不是直接docker-compose up -d
./install.sh
docker-compose ps
运行报另一个错
Harbor docker login x509 certificate signed by unknown authority
docker login --username=admin allinone.xxx.com -p Harbor12345
Error response from daemon: Get "https://harbor.st.com:4443/v2/": x509: certificate signed by unknown authority
原因是Docker未信任我们自己生成的CA证书
|
文件名 |
说明 |
|
ca.crt |
CA证书公钥 |
|
ca.key |
CA证书密钥 |
|
harbor.crt |
Harbor SSL证书公钥 |
|
harbor.key |
Harbor SSL证书密钥 |
方法一
# 192.168.37.170为Habor hostname
mkdir -p /etc/docker/certs.d/192.168.37.170
cp ca.crt /etc/docker/certs.d/192.168.37.170
systemctl restart docker
如果Docker daemon的方法不行，则参考下面的方法在操作系统级别导入并信任我们生成的CA证书。
方法二实测成功
cp ca.crt /etc/pki/ca-trust/source/anchors
update-ca-trust extract
systemctl restart docker
注意：是ca.crt，不是harbor.crt
方法三
cp ca.crt /etc/pki/tls/certs/ca.crt
# backup ca-bundle.crt
cp -p ca-bundle.crt ca-bundle.crt.bak
cat ca.crt >> /etc/pki/tls/certs/ca-bundle.crt
systemctl restart docker
注意：是ca.crt，不是harbor.crt
另一篇参考文档
https://github.com/goharbor/harbor/issues/6774
不同系统添加证书
来自 <https://blog.csdn.net/SHELLCODE_8BIT/article/details/125250740>
Mac OS X
添加证书：
sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain ~/new-root-certificate.crt
移除证书：
sudo security delete-certificate -c ""
Windows
添加证书：
certutil -addstore -f "ROOT" new-root-certificate.crt
移除证书：
certutil -delstore "ROOT" serial-number-hex
Linux (Ubuntu, Debian)
添加证书：
1.复制 CA 文件到目录： /usr/local/share/ca-certificates/
2.执行:
sudo cp foo.crt /usr/local/share/ca-certificates/foo.crt
3.更新 CA 证书库:
sudo update-ca-certificates
移除证书：
1.Remove your CA.
2.Update the CA store:
sudo update-ca-certificates --fresh
Restart Kerio Connect to reload the certificates in the 32-bit versions or Debian 7.
Linux (CentOs 6)
添加证书：
1.安装 ca-certificates package:
yum install ca-certificates
2.启用dynamic CA configuration feature:
update-ca-trust force-enable
3.Add it as a new file to /etc/pki/ca-trust/source/anchors/:
cp foo.crt /etc/pki/ca-trust/source/anchors/
4.执行:
update-ca-trust extract
Restart Kerio Connect to reload the certificates in the 32-bit version.
构建自己的jenkins-docker 增加ca证书
[root@localhost dockerdind]# cat Dockerfile
FROM docker:dind
COPY ca.crt /usr/local/share/ca-certificates/
RUN update-ca-certificates
docker build -t docker:mydind .
docker run \
--name jenkins-docker \
--detach \
--privileged \
--network jenkins \
--network-alias docker \
--env DOCKER_TLS_CERTDIR=/certs \
--volume jenkins-docker-certs:/certs/client \
--volume jenkins-data:/var/jenkins_home \
--publish 2376:2376 \
--publish 3000:3000 --publish 5000:5000 \
docker:mydind \
--storage-driver overlay2
至此，Jenkins docker版用https登录harbor成功
