jenkins整合码云和阿里云私有镜像,k8s部署和可视化

1. cicd流水线

版本

Docker 20.10.7

Harbor 2.3.0

https://github.com/goharbor/harbor/releases/tag/v2.3.0

Jenkins 2.289.1

Redhat Jenkins Packages

https://archives.jenkins-ci.org/windows-stable/2.289.1/

GitLab 14.3.2

https://packages.gitlab.com/app/gitlab/gitlab-ce/search?q=14.3.2&filter=all&filter=all&dist=

单节点服务器要求:

2核,8g,40gb

或者

Docker 20.10.7

Jenkins 2.289.1

阿里云私人镜像库

码云

1.1 docker安装

yum install -y yum-utils device-mapper-persistent-data lvm2

yum-config-manager --add-repo http://mirrors.aliyun.com/dockerce/linux/centos/docker-ce.repo

yum makecache fast

sudo yum install -y yum-utils
sudo yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

yum list docker-ce --showduplicates | sort -r


sudo yum install -y docker-ce-20.10.7 docker-ce-cli-20.10.7 containerd.io


sudo systemctl start docker
sudo systemctl enable docker

sudo docker --version




关闭docker

sudo systemctl stop docker.socket
sudo systemctl stop docker.service


sudo systemctl status docker

设置镜像库

mkdir -p /etc/docker

tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": [
    "https://dockerpull.com",
    "https://dockerproxy.net",
    "https://docker.m.daocloud.io",
    "https://docker.ketches.cn",
    "https://do.nark.eu.org"
  ]
}
EOF

systemctl daemon-reload
systemctl restart docker

systemctl enable docker

docker -v
docker version
docker info


1.2 Harbor(HTTPS)安装(如果服务器顶不住,可以换成阿里云私人镜像仓库)

解压harbor

echo "192.168.49.159 harbor.lagouedu.com" >> /etc/hosts

cat /etc/hosts

cd /data

tar zxf harbor-offline-installer-v2.3.0.tgz

cd harbor/

mkdir -p ssl

cd ssl


获得证书颁发机构

在生产环境中,应该从CA官方获取证书。在测试或开发环境中,可以生成自己的CA。若要生成CA证
书,请运行以下命令。

cd /data/harbor/ssl

创建CA根证书
openssl genrsa -out ca.key 4096


openssl req -x509 -new -nodes -sha512 -days 3650 -subj "/C=TW/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=harbor.lagouedu.com" -key ca.key -out ca.crt

获取服务器证书
openssl genrsa -out harbor.lagouedu.com.key 4096

openssl req -sha512 -new -subj \
"/C=TW/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=harbor.lagouedu.com" \
-key harbor.lagouedu.com.key -out harbor.lagouedu.com.csr

cat > v3.ext <<-EOF
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1=harbor.lagouedu.com
EOF


openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key \
-CAcreateserial -in harbor.lagouedu.com.csr -out harbor.lagouedu.com.crt


openssl x509 -inform PEM -in harbor.lagouedu.com.crt -out harbor.lagouedu.com.cert

mkdir -p /etc/docker/certs.d/harbor.lagouedu.com/

cp harbor.lagouedu.com.cert /etc/docker/certs.d/harbor.lagouedu.com/
cp harbor.lagouedu.com.key /etc/docker/certs.d/harbor.lagouedu.com/
cp ca.crt /etc/docker/certs.d/harbor.lagouedu.com/

systemctl daemon-reload
systemctl restart docker







vi harbor.yml

注释掉http的配置信息

hostname: harbor.lagouedu.com
https:
port: 443
certificate: /data/harbor/ssl/harbor.lagouedu.com.crt
private_key: /data/harbor/ssl/harbor.lagouedu.com.key

安装harbor

docker pull goharbor/prepare:v2.3.0
cd /data/harbor
./prepared
./install.sh

访问UI

C:\Windows\System32\drivers\etc
192.168.198.101 harbor.lagouedu.com
https://harbor.lagouedu.com/

上传镜像

docker-100服务器:
将harbor服务器端生成的ca.crt文件复制到/etc/pki/ca-trust/source/anchors/中。
执行命令更新ca证书授权:update-ca-trust
重启docker服务:
systemctl restart docker
echo "192.168.49.159 harbor.lagouedu.com" >> /etc/hosts
docker login harbor.lagouedu.com
admin
Harbor12345

docker load -i nginx.1.19.3.alpine.tar
docker tag nginx:1.19.3-alpine harbor.lagouedu.com/lagouedu/nginx:v1
docker push harbor.lagouedu.com/lagouedu/nginx:v1

1.3 Harbor(HTTP)安装(如果服务器顶不住,可以换成阿里云私人镜像仓库)

解压harbor

echo "192.168.49.159 harbor.lagouedu.com" >> /etc/hosts

cat /etc/hosts

cd /data

tar zxf harbor-offline-installer-v2.3.0.tgz

cd harbor/

修改配置文件
vi harbor.yml


修改私服镜像地址
hostname: 192.168.49.159
修改镜像地址访问端口号
port: 5000
harbor管理员登录系统密码
harbor_admin_password: Harbor12345
修改harbor映射卷目录
data_volume: /data/harbor


安装harbor
执行启动脚本,经过下述3个步骤后,成功安装harbor私服
./install.sh
准备安装环境:检查docker版本和docker-compose版本
加载harbor需要的镜像
准备编译环境
启动harbor。通过docker-compose方式启动服务

google浏览器访问harbor私服

http://192.168.49.159:5000
username: admin
password: Harbor12345


启动,关闭harbor

启动
docker-compose up -d
关闭
docker-compose down

配置harbor私服

jenkinsagent-154服务器配置docker登录harbor私服信息。
配置私服

vi /etc/docker/daemon.json

"insecure-registries":["192.168.49.159:5000"]

重启docker服务:

systemctl daemon-reload
systemctl restart docker

1.4 GitLab安装(如果服务器顶不住,可以换成码云或者GitHub)

安装

yum -y install policycoreutils openssh-server openssh-clients postfix

可以选择下载成rpm包到指定路径,不安装

yum install --downloadonly --downloaddir=/path/to/download policycoreutils openssh-server openssh-clients postfix
systemctl enable sshd && sudo systemctl start sshd
systemctl enable postfix && systemctl start postfix
rpm -i gitlab-ce-14.3.2-ce.0.el7.x86_64.rpm
vim  /etc/gitlab/gitlab.rb

修改gitlab访问地址和端口,默认为80,我们不进行修改。
external_url 'http://192.168.66.152'
//external_url 'http://<你的服务器地址或域名>:11000'

gitlab-ctl reconfigure
gitlab-ctl restart

启动和停止

#启动服务
# gitlab-ctl start
#停止服务
# gitlab-ctl stop
#重启服务
# gitlab-ctl restart
#状态
#gitlab-ctl status
#监控
#gitlab-ctl  tailunicorn 监控unicorn日志
#gitlab-ctl  tail

登录gitlab

登录gitlab:用户名默认为root。第一次登录需要设置密码。本教程将密码设置为12345678

username:root
password:12345678

1.5 Jenkins安装(Linux)

https://www.oracle.com/java/technologies/downloads/#java8

安装jenkins,jdk,git,maven

需要在jenkins上安装相应的jdk,git,maven

sudo yum install -y curl-devel expat-devel gettext-devel openssl-devel zlib-devel gcc perl-ExtUtils-MakeMaker

sudo yum -y remove git

sudo yum install wget

wget https://download.oracle.com/otn/java/jdk/8u261-b12/a4634525489241b9a9e1aa73d9e118e6/jdk-8u421-linux-x64.tar.gz

wget https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.6.3/apache-maven-3.6.3-bin.tar.gz

wget https://www.kernel.org/pub/software/scm/git/git-2.28.0.tar.gz

tar -zxvf jdk-8u421-linux-x64.tar.gz -C /opt

tar -zxf apache-maven-3.6.3-bin.tar.gz -C /opt
mv /opt/apache-maven-3.6.3 /opt/maven

tar -zxvf git-2.28.0.tar.gz

cd git-2.28.0
./configure --prefix=/opt/git
make && sudo make install

vi /etc/profile
export PATH
export JAVA_HOME=/opt/jdk1.8.0_421
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
export MAVEN_HOME=/opt/maven
export PATH=$MAVEN_HOME/bin:$PATH
export PATH=$PATH:/opt/git/bin

source /etc/profile

java -version

mvn -version

git --version

ln -s /opt/jdk1.8.0_421/bin/java /usr/bin/
ln -s /opt/maven/bin/mvn /usr/bin/
ln -s /opt/git/bin/git /usr/bin

mkdir -p /data/maven/repository

c

设置本地仓库目录
<localRepository>/data/maven/repository</localRepository>

<mirror>
    <id>nexus-aliyun</id>
    <mirrorOf>*</mirrorOf>
    <name>Nexus aliyun</name>
    <url>http://maven.aliyun.com/nexus/content/groups/public</url>
</mirror>


maven工程JDK8编译配置


<profile>
    <id>jdk-1.8</id>
    <activation>
        <activeByDefault>true</activeByDefault>
        <jdk>1.8</jdk>
    </activation>
    <properties>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
        <maven.compiler.compilerVersion>1.8</maven.compiler.compilerVersion>
    </properties>
</profile>

需要开启全局代理,部分命令无法开启,也可以使用 proxychains 配置 Docker 和 Jenkins 等软件的命令

地址是windows宿主机代理的网络地址和可访问ip

sudo vi /etc/profile

export http_proxy="http://root:root@192.168.1.104:9530"
export https_proxy="http://root:root@192.168.1.104:9530"
export ftp_proxy="http://root:root@192.168.1.104:9530"


source /etc/profile

curl http://www.google.com
wget http://www.google.com



上传centos服务器进行安装:
rpm -ivh jenkins-2.289.1-1.1.noarch.rpm

whereis jenkins

vi /etc/init.d/jenkins
img

/opt/jdk1.8.0_421/bin/java

修改Jenkins配置

vi /etc/sysconfig/jenkins

修改内容如下:

JENKINS_USER="root"
JENKINS_PORT="8888"

JENKINS_JAVA_OPTIONS="-Dhudson.model.DownloadService.noSignatureCheck=true"

修改Jenkins插件安装版本为当前版本

或者在图形界面上设置,参照

https://www.haxi.cc/2022/11/

https://mirrors.jenkins.io/updates/dynamic-stable-2.289.1/update-center.json?skipSignatureCheck=true

vi /var/lib/jenkins/hudson.model.UpdateCenter.xml

编辑文件: 打开文件并将 <url> 字段替换为新地址:

<sites>
  <site>
    <id>default</id>
    <url>https://mirrors.jenkins.io/updates/dynamic-stable-2.289.1/update-center.json</url>
  </site>
</sites>

启动Jenkins服务

sudo systemctl daemon-reload

systemctl start jenkins
systemctl enable jenkins

停止jenkins

systemctl stop jenkins

打开浏览器访问
http://192.168.49.159:8888

获取密码

获取并输入admin账户密码
cat /var/lib/jenkins/secrets/initialAdminPassword

root

插件安装

浏览器访问
http://192.168.49.159:8888/jnlpJars/jenkins-cli.jar

在界面开启

image-20241228201942510

java -jar /opt/jenkins-cli.jar -s http://192.168.49.159:8888 \
install-plugin workflow-aggregator pipeline-stage-step docker-plugin docker-workflow \
credentials credentials-binding git gitlab-plugin \
gitee mailer email-ext timestamper matrix-auth thinBackup \
ssh-slaves build-timeout  generic-webhook-trigger



tail -n 100 /var/log/jenkins/jenkins.log

日志最后没有继续输出后,重启Jenkins

systemctl restart jenkins

1.6 Jenkins安装(windows)

https://www.jenkins.io/download/thank-you-downloading-windows-installer-stable/

https://www.yangshaofeng.com/home/Detail?id=3364CF3F300A4E0E8A21D2FA26848176

需要提前在windows上安装好jdk,maven,git,并开启vpn

安装jenkins.msi包

进入安装目录,修改jenkins.xml 配置文件

访问jenkins

http://localhost:8080

选择默认插件安装

1.7 码云和阿里云镜像仓库创建

码云仓库

登录码云,新建git仓库

image-20241226172735291

私人镜像库

1.首先登陆阿里云,进入控制台,打开主菜单,找到“容器镜像服务”

img

2.仓库实例有个人版和企业版,个人版是免费,企业版收费

img

3.进入个人实例,可以看到这个仓库的摘要信息

img

4.进入仓库,这个时候镜像为空,你可以自己push上来docker镜像

img

5.可以给镜像用命名空间进行分类,个人版命名空间限制3个也够用了

img

6.在本机把我自己的进项上传上来,命令很简单,主要docker login 和docker push

img

7.还可以查看镜像的详情

img

1.8 脚本编写

点进去

image-20241228221400374

image-20241228221427368

编写pipeline脚本

image-20241228232043889

码云用户名和密码

image-20241228232019984

新增凭据

image-20241228234431105

image-20241228232638392

gitee的api令牌:

05bf6dbb7ea13e272c6d05a89e4dff5d

进入片段生成器,

image-20241228235838804

生成groovy语句

将生成的拉取语句放到流水线脚本中

image-20241229000048300

(gitlab)版本

设置git提交,自动触发Jenkinsflie执行,jenkinsfile在根目录下

image-20241229005213158

img

img

img

img

(gitee)版本

image-20241230151450144

image-20241230212902171

image-20241230212931214

image-20241230213029763

image-20241230213000464

1.9 推送基础镜像到阿里云

docker pull openjdk:8-alpine3.9

docker login registry.cn-hangzhou.aliyuncs.com -u aliyun0291282216 -p Citygis@1613

docker tag openjdk:8-alpine3.9 registry.cn-hangzhou.aliyuncs.com/dddd56656/openjdk:8-alpine3.9

docker push registry.cn-hangzhou.aliyuncs.com/dddd56656/openjdk:8-alpine3.9

1.10 测试能否成功打包

mvn clean package -Dmaven.test.skip=true jib:build -DsendCredentialsOverHttp=true

1.11 测算代码提交触发容器运行

删除多余路径,修改为指定路径

image-20241230145348090

提交代码,触发jenkins执行pipeline

2. k8s部署和编排

2.1 k8s部署

master节点至少2核2G

不使用rancher,rancher更新跟不上进度

Kubernetes Dashboard v2.8.2

sealos_5.0.0-beta4_linux_amd64.tar.gz
kubernetes 1.27.10

https://github.com/labring/sealos/releases/tag/v5.0.0-beta4

https://github.com/kubernetes/kubernetes/tags?after=v1.30.0-alpha.3

新增三台节点

192.168.49.159 linux159(原docker的cicd服务器)

新增的三台k8s节点

192.168.49.160 linux160
192.168.49.161 linux161
192.168.49.162 linux162

设置集群

systemctl stop firewalld && systemctl disable firewalld
systemctl stop NetworkManager && systemctl disable NetworkManager

setenforce 0
sed -i s/SELINUX=enforcing/SELINUX=disabled/ /etc/selinux/config

swapoff -a
sed -ri 's/.swap./#&/' /etc/fstab

yum install chrony -y
systemctl enable chronyd --now
chronyc sources

升级内核(旧版本只能手动下载)

yum install -y wget

wget http://mirrors.coreix.net/elrepo-archive-archive/kernel/el7/x86_64/RPMS/kernel-lt-devel-5.4.203-1.el7.elrepo.x86_64.rpm
wget http://mirrors.coreix.net/elrepo-archive-archive/kernel/el7/x86_64/RPMS/kernel-lt-headers-5.4.203-1.el7.elrepo.x86_64.rpm
wget http://mirrors.coreix.net/elrepo-archive-archive/kernel/el7/x86_64/RPMS/kernel-lt-5.4.203-1.el7.elrepo.x86_64.rpm

yum install -y perl

rpm -Uvh *.rpm

rpm -qa | grep kernel

awk -F' '$1=="menuentry " {print $2}' /etc/grub2.cfg

grub2-set-default 0

所有服务器都要

拿sealos_5.0.0-beta4_linux_amd64.tar.gz里面的sealos,授权并移动到/usr/bin目录中

cd /opt

chmod +x sealos && mv sealos /usr/bin

单master多node:

sealos所在服务器需要

sealos pull registry.cn-shanghai.aliyuncs.com/labring/kubernetes:v1.27.10 registry.cn-shanghai.aliyuncs.com/labring/helm:v3.9.4 registry.cn-shanghai.aliyuncs.com/labring/cilium:v1.14.7

sealos run --force registry.cn-shanghai.aliyuncs.com/labring/kubernetes:v1.27.10 registry.cn-shanghai.aliyuncs.com/labring/helm:v3.9.4 registry.cn-shanghai.aliyuncs.com/labring/cilium:v1.14.7
--masters 192.168.49.160
--nodes 192.168.49.161,192.168.49.162 -u root -p 123456

如果失败: sealos reset --force

2.2 dashboard安装和连接k8s

Kubernetes Dashboard v2.7.0

helm repo add kubernetes-dashboard https://kubernetes.github.io/dashboard/
helm repo update

helm show chart kubernetes-dashboard/kubernetes-dashboard

本地环境

卸载

helm uninstall kubernetes-dashboard --namespace kube-system

helm install kubernetes-dashboard kubernetes-dashboard/kubernetes-dashboard \
  --version 6.0.8 \
  --namespace kube-system \
  --set replicaCount=1 \
  --set service.port=443 \
  --set service.targetPort=8443 \
  --set service.type=NodePort \
  --set service.nodePort=30001 \
  --set image.repository=dockerproxy.net/kubernetesui/dashboard \--set image.name=dashboard \
  --set image.tag=v2.7.0 \
  --set image.pullPolicy=IfNotPresent

kubectl get pods -n kube-system

kubectl describe pod kubernetes-dashboard-5948b5f5d7-whlqw -n kube-system

kubectl get svc -n kube-system

kubectl create sa dashboard -n kube-system
kubectl create clusterrolebinding dashboard-cluster-admin \
  --clusterrole=cluster-admin \
  --serviceaccount=kube-system:dashboard

echo "
apiVersion: v1
kind: Secret
metadata:
  name: dashboard-sec
  namespace: kube-system
  annotations:
    kubernetes.io/service-account.name: \"dashboard\"
type: kubernetes.io/service-account-token
" > dashboard-sec.yaml

kubectl apply -f dashboard-sec.yaml


kubectl get secret -n kube-system


kubectl describe secret/dashboard-sec -n kube-system | tail -n 1

2.1 dashboard使用

创建命名空间

image-20250105015246892

posted @   惊世智慧  阅读(74)  评论(0编辑  收藏  举报
相关博文:
阅读排行:
· 微软正式发布.NET 10 Preview 1:开启下一代开发框架新篇章
· 没有源码,如何修改代码逻辑?
· PowerShell开发游戏 · 打蜜蜂
· 在鹅厂做java开发是什么体验
· WPF到Web的无缝过渡:英雄联盟客户端的OpenSilver迁移实战
点击右上角即可分享
微信分享提示