基于kubernetes构建jenkins+gitlab持续集成

@

目录

• 安装jenkins
• 安装gitlab
• jenkins连接gitlab
• 配置Jenkins连接maven
• 配置cicd

本案例使用docker-compose来安装部署Jenkins和gitlab

节点	IP
master	192.168.200.44
node	192.168.200.45

安装jenkins

解压安装包

tar -zxvf CICD_Offline.tar

上传安装包忠的镜像到docker

docker load -i jenkins.tar

创建一个jenkins目录，然后编写compose文件

mkdir jenkins

cd jenkins/

vim docker-compose.yaml

version: '3.1'
services:
  jenkins:
    image: 'jenkins/jenkins:2.262-centos'
    volumes:
      - /home/jenkins_home:/var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock
      - /usr/bin/docker:/usr/bin/docker
      - /usr/bin/kubectl:/usr/local/bin/kubectl
      - /root/.kube:/root/.kube
    ports:
      - "8080:8080"
    expose:
      - "8080"
      - "50000"
    privileged: true
    user: root
    restart: always
    container_name: jenkins

启动jenkins

docker-compose up -d

安装插件，重启docker

cp -rfv /root/plugins/* /home/jenkins_home/plugins/

docker restart jenkins

浏览器访问jenkins；http://192.168.200.44:8080/

#容器中查看Jenkins密码

docker exec jenkins cat /var/jenkins_home/secrets/initialAdminPassword

不要安装插件，也不用联网

进入后创建一个Jenkins用户

1，点击系统管理后，点击管理用户
2，点击新建一个用户

3，然后退出重新登录即可

安装gitlab

root目录下创建一个gitlab目录，并配置gitlab

mkdir gitlab

cd gitlab/

vim docker-compose.yaml

version: '3'
services:
  gitlab:
    image: 'gitlab/gitlab-ce:12.9.2-ce.0'
    container_name: gitlab
    restart: always
    privileged: true
    environment:
      TZ: 'Asia/Shanghai'
    ports:
      - '81:80'
      - '443:443'
      - '1022:22'
    volumes:
      - /srv/gitlab/config:/etc/gitlab
      - /srv/gitlab/gitlab/logs:/var/log/gitlab
      - /srv/gitlab/gitlab/data:/var/opt/gitlab

启动gitlab

docker-compose up -d

界面登录；192.168.200.44:81

注：第一次登录需要手动设置密码，密码应该要复杂一点8位，不然设置不了

然后创建一个新的项目，等级要设置为公开

创建好项目后，把本地的springcloud源码git到项目中

cd springcloud/

git config --global user.name "administrator"

git config --global user.email "admin@example.com"

git remote remove origin

git remote add origin http://192.168.200.44:81/root/springcloud.git

git add .

git commit -m "initial commit"

git push -u origin master

输入用户名密码后，网页刷新即可查看

jenkins连接gitlab

点击扳手图标后，点击设置（setting），点击网络（networking）

设置“Outbound requests”，勾选“Allow requests to the local network from web hooks and services”

创建Gitlab API Token

点击Gitlab用户头像图标后点击setting

创建密钥并复制，注意设置令牌到期2时间日期要大于当天


Jenkins点击系统管理，然后点击全局配置，找到连接gitlab，配置完毕后点击测试连接按钮，然后出现404错误，检查密钥是否过期

配置Jenkins连接maven

由于Jenkins是采用docker in docker的方式启动的，所以需要在jenkins容器内安装maven

将maven传到容器中

cp -rfv apache-maven-3.6.3-bin.tar.gz /home/jenkins_home/

进入容器后解压

docker exec -it jenkins bash

tar -zxvf /var/jenkins_home/apache-maven-3.6.3-bin.tar.gz -C .

将maven移动到/usr/local/下

mv apache-maven-3.6.3/ /usr/local/maven

配置maven环境变量，添加至profile末尾

vi /etc/profile

export M2_HOME=/usr/local/maven
export PATH=$PATH:$M2_HOME/bin

查看maven是否安装成功

root@9a9947295632 ~]# source /etc/profile
[root@9a9947295632 ~]# mvn -v
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /usr/local/maven
Java version: 1.8.0_265, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.265.b01-0.el8_2.x86_64/jre
Default locale: en_US, platform encoding: ANSI_X3.4-1968
OS name: "linux", version: "3.10.0-1160.el7.x86_64", arch: "amd64", family: "unix"
[root@9a9947295632 ~]#

将maven加入开机自启

[root@9a9947295632 ~]# vi /root/.bashrc
[root@9a9947295632 ~]# cat /root/.bashrc
# .bashrc

# User specific aliases and functions

alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'

# Source global definitions
if [ -f /etc/bashrc ]; then
        . /etc/bashrc
        source /etc/profile
fi

连接maven，登录Jenkins首页，点击“系统管理”→“全局工具配置”，点击“新增Maven”，如图所示。取消勾选“自动安装”，填入maven名称和安装路径，配置完成后点击“应用”

配置cicd

录Jenkins首页，点击左侧导航栏“新建任务”，如图所示，选择构建一个流水线

配置构建触发器，并记录下GitLab webhook URL的地址（http://192.168.200.44:8080/project/springcloud），后期配置webhook需要使用

配置流水线，点击“流水线语法”，如图所示，示例步骤选择“git：Git”，将springcloud项目地址填入仓库URL，点击“添加”→“jenkins”添加凭据，如图所示。类型选择“Username with password”，用户名和密码为Gitlab仓库的用户名和密码

点击“生成流水线脚本”

将以下流水线脚本写入网页中点击应用

node{

    stage('git clone'){
        //check CODE
        git credentialsId: '294160aa-17b8-4536-9db1-d71ddccd4b93', url: 'http://192.168.200.44:81/root/springcloud.git'
    }
    stage('maven build'){
        sh '''/usr/local/maven/bin/mvn package -DskipTests -f /var/jenkins_home/workspace/springcloud'''
    }
    stage('image build'){
        sh '''
              echo $BUILD_ID
              docker build -t 192.168.200.44/springcloud/gateway:$BUILD_ID -f /var/jenkins_home/workspace/springcloud/gateway/Dockerfile  /var/jenkins_home/workspace/springcloud/gateway
              docker build -t 192.168.200.44/springcloud/config:$BUILD_ID -f /var/jenkins_home/workspace/springcloud/config/Dockerfile  /var/jenkins_home/workspace/springcloud/config'''
    }
    stage('test'){
        sh '''docker run -itd --name gateway 192.168.200.44/springcloud/gateway:$BUILD_ID
        docker ps -a|grep springcloud|grep Up
        if [ $? -eq 0 ];then
            echo "Success!"
            docker rm -f gateway
        else
            docker rm -f gateway
            exit 1
            fi

        '''
    }
    stage('upload registry'){
        sh '''docker login 192.168.200.44 -u=admin -p=Harbor12345
            docker push 192.168.200.44/springcloud/gateway:$BUILD_ID
            docker push 192.168.200.44/springcloud/config:$BUILD_ID'''
    }
    stage('deploy Rancher'){
        //执行部署脚本
       sh 'sed -i "s/sqshq\\/piggymetrics-gateway/192.168.200.44\\/springcloud\\/gateway:$BUILD_ID/g" /var/jenkins_home/workspace/springcloud/yaml/deployment/gateway-deployment.yaml'
       sh 'sed -i "s/sqshq\\/piggymetrics-config/192.168.200.44\\/springcloud\\/config:$BUILD_ID/g" /var/jenkins_home/workspace/springcloud/yaml/deployment/config-deployment.yaml'
       sh 'kubectl create ns springcloud'
       sh 'kubectl apply -f /var/jenkins_home/workspace/springcloud/yaml/deployment/gateway-deployment.yaml --kubeconfig=/root/.kube/config'
       sh 'kubectl apply -f /var/jenkins_home/workspace/springcloud/yaml/deployment/config-deployment.yaml --kubeconfig=/root/.kube/config'
       sh 'kubectl apply -f /var/jenkins_home/workspace/springcloud/yaml/svc/gateway-svc.yaml --kubeconfig=/root/.kube/config'
       sh 'kubectl apply -f /var/jenkins_home/workspace/springcloud/yaml/svc/config-svc.yaml --kubeconfig=/root/.kube/config'

    }
}

开启Jenkins匿名访问

登录Jenkins首页，点击“系统管理”→“全局安全配置”，配置授权策略允许匿名用户访问

配置Webhook

登录Gitlab，进入springcloud项目，点击左侧导航栏“Settings”→“Webhooks”，将前面记录的GitLab webhook URL地址填入URL处，禁用SSL认证

点击“Add webhook”添加webhook，然后点击“Test”→“Push events”进行测试

结果返回HTTP 200则表明Webhook配置成功

创建仓库项目

登录Harbor，新建项目springcloud，访问级别设置为公开（admin；Harbor12345）

触发CI/CD

上传代码触发自动构建

docker cp repository/ jenkins:/root/.m2/

docker restart jenkins

cd springcloud/

git add .

git commit -m "Initial commit"

一般自动构建都会失败，去界面构建

脚本执行到最后一部分出错

执行到流水线的此处脚本报错

	   sh 'kubectl create ns springcloud'
       sh 'kubectl apply -f /var/jenkins_home/workspace/springcloud/yaml/deployment/gateway-deployment.yaml --kubeconfig=/root/.kube/config'
       sh 'kubectl apply -f /var/jenkins_home/workspace/springcloud/yaml/deployment/config-deployment.yaml --kubeconfig=/root/.kube/config'
       sh 'kubectl apply -f /var/jenkins_home/workspace/springcloud/yaml/svc/gateway-svc.yaml --kubeconfig=/root/.kube/config'
       sh 'kubectl apply -f /var/jenkins_home/workspace/springcloud/yaml/svc/config-svc.yaml --kubeconfig=/root/.kube/config'

尝试手动执行，会报同样错误

镜像都是部署完毕

应该是k8s的问题，jenkins内部不能使用现有k8s的命令，连接不上，应该要使用国基的k8s部署方式，我使用的是一道云kubeeasy部署的k8s