jenkins整合码云和阿里云私有镜像,k8s部署和可视化
1. cicd流水线
版本
Docker 20.10.7
Harbor 2.3.0
https://github.com/goharbor/harbor/releases/tag/v2.3.0
Jenkins 2.289.1
https://archives.jenkins-ci.org/windows-stable/2.289.1/
GitLab 14.3.2
https://packages.gitlab.com/app/gitlab/gitlab-ce/search?q=14.3.2&filter=all&filter=all&dist=
单节点服务器要求:
2核,8g,40gb
或者
Docker 20.10.7
Jenkins 2.289.1
阿里云私人镜像库
码云
1.1 docker安装
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo http://mirrors.aliyun.com/dockerce/linux/centos/docker-ce.repo
yum makecache fast
sudo yum install -y yum-utils
sudo yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum list docker-ce --showduplicates | sort -r
sudo yum install -y docker-ce-20.10.7 docker-ce-cli-20.10.7 containerd.io
sudo systemctl start docker
sudo systemctl enable docker
sudo docker --version
关闭docker
sudo systemctl stop docker.socket
sudo systemctl stop docker.service
sudo systemctl status docker
设置镜像库
mkdir -p /etc/docker
tee /etc/docker/daemon.json <<-'EOF'
{
"registry-mirrors": [
"https://dockerpull.com",
"https://dockerproxy.net",
"https://docker.m.daocloud.io",
"https://docker.ketches.cn",
"https://do.nark.eu.org"
]
}
EOF
systemctl daemon-reload
systemctl restart docker
systemctl enable docker
docker -v
docker version
docker info
1.2 Harbor(HTTPS)安装(如果服务器顶不住,可以换成阿里云私人镜像仓库)
解压harbor
echo "192.168.49.159 harbor.lagouedu.com" >> /etc/hosts
cat /etc/hosts
cd /data
tar zxf harbor-offline-installer-v2.3.0.tgz
cd harbor/
mkdir -p ssl
cd ssl
获得证书颁发机构
在生产环境中,应该从CA官方获取证书。在测试或开发环境中,可以生成自己的CA。若要生成CA证
书,请运行以下命令。
cd /data/harbor/ssl
创建CA根证书
openssl genrsa -out ca.key 4096
openssl req -x509 -new -nodes -sha512 -days 3650 -subj "/C=TW/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=harbor.lagouedu.com" -key ca.key -out ca.crt
获取服务器证书
openssl genrsa -out harbor.lagouedu.com.key 4096
openssl req -sha512 -new -subj \
"/C=TW/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=harbor.lagouedu.com" \
-key harbor.lagouedu.com.key -out harbor.lagouedu.com.csr
cat > v3.ext <<-EOF
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1=harbor.lagouedu.com
EOF
openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key \
-CAcreateserial -in harbor.lagouedu.com.csr -out harbor.lagouedu.com.crt
openssl x509 -inform PEM -in harbor.lagouedu.com.crt -out harbor.lagouedu.com.cert
mkdir -p /etc/docker/certs.d/harbor.lagouedu.com/
cp harbor.lagouedu.com.cert /etc/docker/certs.d/harbor.lagouedu.com/
cp harbor.lagouedu.com.key /etc/docker/certs.d/harbor.lagouedu.com/
cp ca.crt /etc/docker/certs.d/harbor.lagouedu.com/
systemctl daemon-reload
systemctl restart docker
vi harbor.yml
注释掉http的配置信息
hostname: harbor.lagouedu.com
https:
port: 443
certificate: /data/harbor/ssl/harbor.lagouedu.com.crt
private_key: /data/harbor/ssl/harbor.lagouedu.com.key
安装harbor
docker pull goharbor/prepare:v2.3.0
cd /data/harbor
./prepared
./install.sh
访问UI
C:\Windows\System32\drivers\etc
192.168.198.101 harbor.lagouedu.com
https://harbor.lagouedu.com/
上传镜像
docker-100服务器:
将harbor服务器端生成的ca.crt文件复制到/etc/pki/ca-trust/source/anchors/中。
执行命令更新ca证书授权:update-ca-trust
重启docker服务:
systemctl restart docker
echo "192.168.49.159 harbor.lagouedu.com" >> /etc/hosts
docker login harbor.lagouedu.com
admin
Harbor12345
docker load -i nginx.1.19.3.alpine.tar
docker tag nginx:1.19.3-alpine harbor.lagouedu.com/lagouedu/nginx:v1
docker push harbor.lagouedu.com/lagouedu/nginx:v1
1.3 Harbor(HTTP)安装(如果服务器顶不住,可以换成阿里云私人镜像仓库)
解压harbor
echo "192.168.49.159 harbor.lagouedu.com" >> /etc/hosts
cat /etc/hosts
cd /data
tar zxf harbor-offline-installer-v2.3.0.tgz
cd harbor/
修改配置文件
vi harbor.yml
修改私服镜像地址
hostname: 192.168.49.159
修改镜像地址访问端口号
port: 5000
harbor管理员登录系统密码
harbor_admin_password: Harbor12345
修改harbor映射卷目录
data_volume: /data/harbor
安装harbor
执行启动脚本,经过下述3个步骤后,成功安装harbor私服
./install.sh
准备安装环境:检查docker版本和docker-compose版本
加载harbor需要的镜像
准备编译环境
启动harbor。通过docker-compose方式启动服务
google浏览器访问harbor私服
http://192.168.49.159:5000
username: admin
password: Harbor12345
启动,关闭harbor
启动
docker-compose up -d
关闭
docker-compose down
配置harbor私服
jenkinsagent-154服务器配置docker登录harbor私服信息。
配置私服
vi /etc/docker/daemon.json
"insecure-registries":["192.168.49.159:5000"]
重启docker服务:
systemctl daemon-reload
systemctl restart docker
1.4 GitLab安装(如果服务器顶不住,可以换成码云或者GitHub)
安装
yum -y install policycoreutils openssh-server openssh-clients postfix
可以选择下载成rpm包到指定路径,不安装
yum install --downloadonly --downloaddir=/path/to/download policycoreutils openssh-server openssh-clients postfix
systemctl enable sshd && sudo systemctl start sshd
systemctl enable postfix && systemctl start postfix
rpm -i gitlab-ce-14.3.2-ce.0.el7.x86_64.rpm
vim /etc/gitlab/gitlab.rb
修改gitlab访问地址和端口,默认为80,我们不进行修改。
external_url 'http://192.168.66.152'
//external_url 'http://<你的服务器地址或域名>:11000'
gitlab-ctl reconfigure
gitlab-ctl restart
启动和停止
#启动服务
# gitlab-ctl start
#停止服务
# gitlab-ctl stop
#重启服务
# gitlab-ctl restart
#状态
#gitlab-ctl status
#监控
#gitlab-ctl tailunicorn 监控unicorn日志
#gitlab-ctl tail
登录gitlab
登录gitlab:用户名默认为root。第一次登录需要设置密码。本教程将密码设置为12345678
username:root
password:12345678
1.5 Jenkins安装(Linux)
https://www.oracle.com/java/technologies/downloads/#java8
安装jenkins,jdk,git,maven
需要在jenkins上安装相应的jdk,git,maven
sudo yum install -y curl-devel expat-devel gettext-devel openssl-devel zlib-devel gcc perl-ExtUtils-MakeMaker
sudo yum -y remove git
sudo yum install wget
wget https://www.kernel.org/pub/software/scm/git/git-2.28.0.tar.gz
tar -zxvf jdk-8u421-linux-x64.tar.gz -C /opt
tar -zxf apache-maven-3.6.3-bin.tar.gz -C /opt
mv /opt/apache-maven-3.6.3 /opt/maven
tar -zxvf git-2.28.0.tar.gz
cd git-2.28.0
./configure --prefix=/opt/git
make && sudo make install
vi /etc/profile
export PATH
export JAVA_HOME=/opt/jdk1.8.0_421
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
export MAVEN_HOME=/opt/maven
export PATH=$MAVEN_HOME/bin:$PATH
export PATH=$PATH:/opt/git/bin
source /etc/profile
java -version
mvn -version
git --version
ln -s /opt/jdk1.8.0_421/bin/java /usr/bin/
ln -s /opt/maven/bin/mvn /usr/bin/
ln -s /opt/git/bin/git /usr/bin
mkdir -p /data/maven/repository
c
设置本地仓库目录
<localRepository>/data/maven/repository</localRepository>
<mirror>
<id>nexus-aliyun</id>
<mirrorOf>*</mirrorOf>
<name>Nexus aliyun</name>
<url>http://maven.aliyun.com/nexus/content/groups/public</url>
</mirror>
maven工程JDK8编译配置
<profile>
<id>jdk-1.8</id>
<activation>
<activeByDefault>true</activeByDefault>
<jdk>1.8</jdk>
</activation>
<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<maven.compiler.compilerVersion>1.8</maven.compiler.compilerVersion>
</properties>
</profile>
需要开启全局代理,部分命令无法开启,也可以使用 proxychains
配置 Docker 和 Jenkins 等软件的命令
地址是windows宿主机代理的网络地址和可访问ip
sudo vi /etc/profile
export http_proxy="http://root:root@192.168.1.104:9530"
export https_proxy="http://root:root@192.168.1.104:9530"
export ftp_proxy="http://root:root@192.168.1.104:9530"
source /etc/profile
curl http://www.google.com
wget http://www.google.com
上传centos服务器进行安装:
rpm -ivh jenkins-2.289.1-1.1.noarch.rpm
whereis jenkins
vi /etc/init.d/jenkins
/opt/jdk1.8.0_421/bin/java
修改Jenkins配置
vi /etc/sysconfig/jenkins
修改内容如下:
JENKINS_USER="root"
JENKINS_PORT="8888"
JENKINS_JAVA_OPTIONS="-Dhudson.model.DownloadService.noSignatureCheck=true"
修改Jenkins插件安装版本为当前版本
或者在图形界面上设置,参照
https://mirrors.jenkins.io/updates/dynamic-stable-2.289.1/update-center.json?skipSignatureCheck=true
vi /var/lib/jenkins/hudson.model.UpdateCenter.xml
编辑文件: 打开文件并将 <url>
字段替换为新地址:
<sites>
<site>
<id>default</id>
<url>https://mirrors.jenkins.io/updates/dynamic-stable-2.289.1/update-center.json</url>
</site>
</sites>
启动Jenkins服务
sudo systemctl daemon-reload
systemctl start jenkins
systemctl enable jenkins
停止jenkins
systemctl stop jenkins
打开浏览器访问
http://192.168.49.159:8888
获取密码
获取并输入admin账户密码
cat /var/lib/jenkins/secrets/initialAdminPassword
root
插件安装
浏览器访问
http://192.168.49.159:8888/jnlpJars/jenkins-cli.jar
在界面开启
java -jar /opt/jenkins-cli.jar -s http://192.168.49.159:8888 \
install-plugin workflow-aggregator pipeline-stage-step docker-plugin docker-workflow \
credentials credentials-binding git gitlab-plugin \
gitee mailer email-ext timestamper matrix-auth thinBackup \
ssh-slaves build-timeout generic-webhook-trigger
tail -n 100 /var/log/jenkins/jenkins.log
日志最后没有继续输出后,重启Jenkins
systemctl restart jenkins
1.6 Jenkins安装(windows)
https://www.jenkins.io/download/thank-you-downloading-windows-installer-stable/
https://www.yangshaofeng.com/home/Detail?id=3364CF3F300A4E0E8A21D2FA26848176
需要提前在windows上安装好jdk,maven,git,并开启vpn
安装jenkins.msi包
进入安装目录,修改jenkins.xml 配置文件
访问jenkins
选择默认插件安装
1.7 码云和阿里云镜像仓库创建
码云仓库
登录码云,新建git仓库
私人镜像库
1.首先登陆阿里云,进入控制台,打开主菜单,找到“容器镜像服务”
2.仓库实例有个人版和企业版,个人版是免费,企业版收费
3.进入个人实例,可以看到这个仓库的摘要信息
4.进入仓库,这个时候镜像为空,你可以自己push上来docker镜像
5.可以给镜像用命名空间进行分类,个人版命名空间限制3个也够用了
6.在本机把我自己的进项上传上来,命令很简单,主要docker login 和docker push
7.还可以查看镜像的详情
1.8 脚本编写
点进去
编写pipeline脚本
码云用户名和密码
新增凭据
gitee的api令牌:
05bf6dbb7ea13e272c6d05a89e4dff5d
进入片段生成器,
生成groovy语句
将生成的拉取语句放到流水线脚本中
(gitlab)版本
设置git提交,自动触发Jenkinsflie执行,jenkinsfile在根目录下
(gitee)版本
1.9 推送基础镜像到阿里云
docker pull openjdk:8-alpine3.9
docker login registry.cn-hangzhou.aliyuncs.com -u aliyun0291282216 -p Citygis@1613
docker tag openjdk:8-alpine3.9 registry.cn-hangzhou.aliyuncs.com/dddd56656/openjdk:8-alpine3.9
docker push registry.cn-hangzhou.aliyuncs.com/dddd56656/openjdk:8-alpine3.9
1.10 测试能否成功打包
mvn clean package -Dmaven.test.skip=true jib:build -DsendCredentialsOverHttp=true
1.11 测算代码提交触发容器运行
删除多余路径,修改为指定路径
提交代码,触发jenkins执行pipeline
2. k8s部署和编排
2.1 k8s部署
master节点至少2核2G
不使用rancher,rancher更新跟不上进度
Kubernetes Dashboard v2.8.2
sealos_5.0.0-beta4_linux_amd64.tar.gz
kubernetes 1.27.10
https://github.com/labring/sealos/releases/tag/v5.0.0-beta4
https://github.com/kubernetes/kubernetes/tags?after=v1.30.0-alpha.3
新增三台节点
192.168.49.159 linux159(原docker的cicd服务器)
新增的三台k8s节点
192.168.49.160 linux160
192.168.49.161 linux161
192.168.49.162 linux162
设置集群
systemctl stop firewalld && systemctl disable firewalld
systemctl stop NetworkManager && systemctl disable NetworkManager
setenforce 0
sed -i s/SELINUX=enforcing/SELINUX=disabled/ /etc/selinux/config
swapoff -a
sed -ri 's/.swap./#&/' /etc/fstab
yum install chrony -y
systemctl enable chronyd --now
chronyc sources
升级内核(旧版本只能手动下载)
yum install -y wget
wget http://mirrors.coreix.net/elrepo-archive-archive/kernel/el7/x86_64/RPMS/kernel-lt-devel-5.4.203-1.el7.elrepo.x86_64.rpm
wget http://mirrors.coreix.net/elrepo-archive-archive/kernel/el7/x86_64/RPMS/kernel-lt-headers-5.4.203-1.el7.elrepo.x86_64.rpm
wget http://mirrors.coreix.net/elrepo-archive-archive/kernel/el7/x86_64/RPMS/kernel-lt-5.4.203-1.el7.elrepo.x86_64.rpm
yum install -y perl
rpm -Uvh *.rpm
rpm -qa | grep kernel
awk -F' '$1=="menuentry " {print $2}' /etc/grub2.cfg
grub2-set-default 0
所有服务器都要
拿sealos_5.0.0-beta4_linux_amd64.tar.gz里面的sealos,授权并移动到/usr/bin目录中
cd /opt
chmod +x sealos && mv sealos /usr/bin
单master多node:
sealos所在服务器需要
sealos pull registry.cn-shanghai.aliyuncs.com/labring/kubernetes:v1.27.10 registry.cn-shanghai.aliyuncs.com/labring/helm:v3.9.4 registry.cn-shanghai.aliyuncs.com/labring/cilium:v1.14.7
sealos run --force registry.cn-shanghai.aliyuncs.com/labring/kubernetes:v1.27.10 registry.cn-shanghai.aliyuncs.com/labring/helm:v3.9.4 registry.cn-shanghai.aliyuncs.com/labring/cilium:v1.14.7
--masters 192.168.49.160
--nodes 192.168.49.161,192.168.49.162 -u root -p 123456
如果失败: sealos reset --force
2.2 dashboard安装和连接k8s
Kubernetes Dashboard v2.7.0
helm repo add kubernetes-dashboard https://kubernetes.github.io/dashboard/
helm repo update
helm show chart kubernetes-dashboard/kubernetes-dashboard
本地环境
卸载
helm uninstall kubernetes-dashboard --namespace kube-system
helm install kubernetes-dashboard kubernetes-dashboard/kubernetes-dashboard \
--version 6.0.8 \
--namespace kube-system \
--set replicaCount=1 \
--set service.port=443 \
--set service.targetPort=8443 \
--set service.type=NodePort \
--set service.nodePort=30001 \
--set image.repository=dockerproxy.net/kubernetesui/dashboard \--set image.name=dashboard \
--set image.tag=v2.7.0 \
--set image.pullPolicy=IfNotPresent
kubectl get pods -n kube-system
kubectl describe pod kubernetes-dashboard-5948b5f5d7-whlqw -n kube-system
kubectl get svc -n kube-system
kubectl create sa dashboard -n kube-system
kubectl create clusterrolebinding dashboard-cluster-admin \
--clusterrole=cluster-admin \
--serviceaccount=kube-system:dashboard
echo "
apiVersion: v1
kind: Secret
metadata:
name: dashboard-sec
namespace: kube-system
annotations:
kubernetes.io/service-account.name: \"dashboard\"
type: kubernetes.io/service-account-token
" > dashboard-sec.yaml
kubectl apply -f dashboard-sec.yaml
kubectl get secret -n kube-system
kubectl describe secret/dashboard-sec -n kube-system | tail -n 1
2.1 dashboard使用
创建命名空间
【推荐】编程新体验,更懂你的AI,立即体验豆包MarsCode编程助手
【推荐】凌霞软件回馈社区,博客园 & 1Panel & Halo 联合会员上线
【推荐】抖音旗下AI助手豆包,你的智能百科全书,全免费不限次数
【推荐】博客园社区专享云产品让利特惠,阿里云新客6.5折上折
【推荐】轻量又高性能的 SSH 工具 IShell:AI 加持,快人一步
· 微软正式发布.NET 10 Preview 1:开启下一代开发框架新篇章
· 没有源码,如何修改代码逻辑?
· PowerShell开发游戏 · 打蜜蜂
· 在鹅厂做java开发是什么体验
· WPF到Web的无缝过渡:英雄联盟客户端的OpenSilver迁移实战