jenkins+gitlab+harbor部署CICD
jenkins相关操作
docker部署jenkins
docker run --name jenkins -d -p 8080:8080 -p 50000:50000 --privileged=true --restart=always -v /jenkins/home:/var/jenkins_home -v /var/run/docker.sock:/var/run/docker.sock -v /usr/bin/docker:/usr/bin/docker jenkins/jenkins:2.387.2-lts
打开jenkins访问页面需要下载的插件
Publish Over SSH
GitLab Plugin
Git Parameter Plug-In
配置k8s可以执行yaml的主机
配置webhook,作用是一旦gitlab仓库代码被提交立马给通知jenkins发送CICD的请求
1.关闭对于project的认证
下载安装maven并放到jenkins所映射的宿主机目录下,然后制定maven home
下载jdk并解压到jenkins所映射的宿主机目录下,然后制定jdk的home目录
创建jenkins pipline流水线项目,点击build when....目的是一旦发现gitlab有代码提交jenkins就执行pipline任务
http://xxxx需要记住,一会要在gitlab中配置
制定gitlab仓库的url,并指定分支,然后pipline脚本指定为jenkinsfile
gitlab相关操作
在gtilab上配置webhook,配置好之后点击test测试一下返回200代表ok
gitlab上这2个地方打钩,否则webhook会失败
在gitlab仓库里面除了有java代码以外,还需要有
1.dockerfiile 2.部署pod的yaml文件 3.Jenkinsfile,注意开头字符必须是大写,否则检测不到Jenkins文件
Jenkinsfile文件内容如下:
需要注意的是如何用publish over ssh插件把部署文件传过去之后,要cd的话必须制定绝对路径
pipeline { agent any environment { image="core.harbor.domain:80/myharbor/myjava:latest" harbor="core.harbor.domain:80" } stages { stage('拉去git代码') { steps { checkout scmGit(branches: [[name: '*/main']], extensions: [], userRemoteConfigs: [[url: 'http://192.168.1.101/root/myjava.git']]) } } stage('maven构建项目') { steps { sh '/var/jenkins_home/maven3.9.1/bin/mvn clean package -DskipTests' } } stage('dockerfile制作镜像') { steps { sh ''' docker login -uadmin -pHarbor12345 $harbor cp target/*.jar ./docker cd docker docker build -t $image ./ ''' } } stage('推送镜像到harbor') { steps { sh 'docker push $image' } } stage('通过publish over ssh通知目标服务器拉取镜像并部署') { steps { sshPublisher(publishers: [sshPublisherDesc(configName: 'test01', transfers: [sshTransfer(cleanRemote: false, excludes: '', execCommand: '''cd /jenkins_data/mypipline01 && kubectl apply -f a.yaml && kubectl rollout restart deploy myjava ''', execTimeout: 120000, flatten: false, makeEmptyDirs: false, noDefaultExcludes: false, patternSeparator: '[, ]+', remoteDirectory: 'mypipline01', remoteDirectorySDF: false, removePrefix: '', sourceFiles: '**/*.yaml')], usePromotionTimestamp: false, useWorkspaceInPromotion: false, verbose: false)]) } } } }
yaml部署文件内容如下:
apiVersion: apps/v1 kind: Deployment metadata: labels: app: myjava name: myjava namespace: default spec: replicas: 3 selector: matchLabels: app: myjava template: metadata: labels: app: myjava spec: imagePullSecrets: - name: harbor containers: - image: core.harbor.domain:80/myharbor/myjava:latest imagePullPolicy: Always name: myjavaweb ports: - containerPort: 8080 protocol: TCP name: http
dockerfile内容如下:
FROM java:openjdk-8u111 WORKDIR /usr/local COPY demo2-0.0.1-SNAPSHOT.jar ./ CMD java -jar demo2-0.0.1-SNAPSHOT.jar
部署ingress-nginx(我的k8s版本是1.22)
kubectl create -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.0/deploy/static/provider/clouddeploy.yaml
查看ingress controller是否正常
[root@ceph1 ~]# kubectl get pods -n ingress-nginx NAME READY STATUS RESTARTS AGE ingress-nginx-admission-create--1-vmgcm 0/1 Completed 0 8d ingress-nginx-admission-patch--1-7nbcr 0/1 Completed 2 8d ingress-nginx-controller-54bfb9bb-f4wf6 1/1 Running 8 (19h ago) 8d
[root@ceph1 ~]# kubectl get svc -n ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ingress-nginx-controller NodePort 10.107.14.210 <none> 80:31485/TCP,443:30185/TCP 8d ingress-nginx-controller-admission ClusterIP 10.99.113.251 <none> 443/TCP 8d
然后部署deploy和svc
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
[root@master myself]# cat mypod.yaml apiVersion: v1 kind: Service metadata: name: service-nginx namespace: default spec: selector: app: mynginx ports: - name: http port: 80 targetPort: 80 --- apiVersion: apps/v1 kind: Deployment metadata: name: mydepoy namespace: default spec: replicas: 5 selector: matchLabels: app: mynginx template: metadata: labels: app: mynginx spec: containers: - name: mycontainer image: lizhaoqwe/nginx:v1 imagePullPolicy: IfNotPresent ports: - name: nginx containerPort: 80
编写ingress规则
[root@ceph1 ~]# cat ingress.yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: service-nginx-ingress namespace: default annotations: nginx.ingress.kubernetes.io/rewrite-target: / spec: ingressClassName: nginx #制定ingressclassname,可以用kubectl get ingressclass查看 rules: - host: mynginx.fengzi.com http: paths: - path: / pathType: Prefix backend: service: name: service-nginx #制定svc的名称 port: number: 80
查看ingress
[root@ceph1 ~]# kubectl get ingress NAME CLASS HOSTS ADDRESS PORTS AGE harbor-ingress nginx core.harbor.domain 10.107.14.210 80 21h harbor-ingress-notary nginx notary.harbor.domain 10.107.14.210 80 21h service-nginx-ingress nginx mynginx.fengzi.com 10.107.14.210 80 8d
有ingress之后再要访问服务就需要域名了,所以,在找一台机器搭建nginx,配置文件内容如下:
harbor的配置文件内容如下:
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
[root@hadoop1 conf.d]# cat core.harbor.domain.conf upstream myk8s { server 192.168.1.101:31485 weight=1; server 192.168.1.102:31485 weight=1; server 192.168.1.103:31485 weight=1; } server { listen 80; server_name core.harbor.domain; location / { proxy_set_header Host $host; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-Proto $scheme; proxy_connect_timeout 5s; client_max_body_size 0; proxy_pass http://myk8s; } }
gitlab的配置内容如下:
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
[root@hadoop1 conf.d]# cat mygitlab.fengzi.com.conf upstream mygitlab { server 192.168.1.101:80 weight=1; } server { listen 80; server_name mygitlab.fengzi.com; location / { proxy_set_header Host $host; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-Proto $scheme; proxy_connect_timeout 5s; client_max_body_size 0; proxy_pass http://mygitlab; } }
部署gitlab
curl -fsSL https://packages.gitlab.cn/repository/raw/scripts/setup.sh | /bin/bash
EXTERNAL_URL="http://0.0.0.0" yum install -y gitlab-jh
部署harbor
helm add repo harbor https://helm.goharbor.io
helm pull harbor/harbor
cd harbor
vim values.yaml
修改values.yaml文件
1.如果不需要https访问,需要改成false
2.如果没有pvc,修改为false
3.配置harbor域名
一般externalURL的域名与上面core的一致
4.制定classname,也就是kubectl get ingressclass的name
5.部署harbor
helm install harbor ../harbor
全都完成以后推送代码后,最终的效果如下:
真实环境的pipline
pipeline { agent any stages { stage('设置环境变量') { steps { script { env.TAG = sh(returnStdout: true, script: "git log | head -1 | awk '{print \$2}' | cut -b '1-8'").trim() env.HARBOR = "xxx.xxx.xxx:8888" env.IMAGE = "avp" env.PROJECT = "avp" env.GITLAB = "http://xx.xx.xx.167/itmp/avp.git" env.PORT = "20041" env.BRANCH = "origin/develop_avp" } } } stage('拉取git代码') { steps { checkout([$class: 'GitSCM', branches: [[name: "${env.BRANCH}"]], extensions: [], userRemoteConfigs: [[credentialsId: 'dd61867f-be2d-4283-ab1c-ab2c5bd0b8ab', url: "${env.GITLAB}"]]]) } } stage('maven构建项目') { steps { sh 'mvn clean package -DskipTests' } } stage('编写dockerfile') { steps { sh '''cat > Dockerfile << "EOF" FROM strongzz/openjdk:alpine-8u342 RUN ln -snf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && echo "Asia/Shanghai" > /etc/timezone ADD ./target/*.jar /app.jar ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"] EOF''' } } stage('制作镜像') { steps { sh "docker build -t ${env.HARBOR}/${env.PROJECT}/${env.IMAGE}:${env.TAG} ./" } } stage('push到harbor仓库') { steps { sh "docker push ${env.HARBOR}/${env.PROJECT}/${env.IMAGE}:${env.TAG}" } } stage('删除本地docker镜像') { steps { sh "docker rmi ${env.HARBOR}/${env.PROJECT}/${env.IMAGE}:${env.TAG}" } } stage('生成部署清单') { steps { sh """cat > ${env.IMAGE}.yaml <<EOF apiVersion: apps/v1 kind: Deployment metadata: name: ${env.IMAGE} namespace: ${env.PROJECT} spec: replicas: 1 selector: matchLabels: app: ${env.IMAGE} template: metadata: labels: app: ${env.IMAGE} spec: containers: - name: ${env.IMAGE} image: ${env.HARBOR}/${env.PROJECT}/${env.IMAGE}:${env.TAG} command: ["java","-jar","/app.jar","--spring.profiles.active=develop"] ports: - containerPort: ${env.PORT} EOF""" } } stage('更新镜像') { steps { sh "export KUBECONFIG=/var/jenkins_home/.kube/jiuzhou-config && kubectl apply -f ${env.IMAGE}.yaml" } } } }