持续集成（jenkins + pipeline + k8s）

目前网上自动化持续集成的资料很多，但基本上都是局限于jenkins自由风格的job，结合shell脚本来实现持续集成，这种方式的缺点也很明显： 
构建出问题，排查困难
构建节点挂了，就不能完成构建任务
而当前主流技术是 “ k8s + 微服务 ” 等，我们完全可以利用k8s的优势来完成持续构建任务，每次构建时可以调度到任意节点，或者是具有指定标签的节点，这就实现了高可用，另外，结合pipeline，可以轻松简单实现持续集成，并且哪个阶段出问题了一目了然，排查起来也容易。

02

自动化测试框架

_____

0基础到实现：java + testng + httpclient + allure参考：https://www.cnblogs.com/uncleyong/p/15867903.html

03

企业级持续集成技术栈

_____

git + gitlab + jenkins + pipeline + maven + harbor + docker + k8s可以整合python、java等各种自动化测试框架流程：拉取代码--》mvn打包--》构建镜像--》新镜像发布到k8s--》拉取自动化测试代码--》自动化测试--》allure报告环境：jenkins使用k8s作为构建环境，在某个节点执行测试

 

 

 

04

环境规划

_____

192.168.117.160：harbor、jenkins、maven

192.168.117.161：k8s-master

192.168.117.162：k8s-node01

192.168.117.163：k8s-node02

192.168.117.180：gitlab

每台虚拟机都安装了git 

05

环境搭建

_____

git安装及使用：https://www.cnblogs.com/uncleyong/p/10854115.html

gitlab安装及使用：https://www.cnblogs.com/uncleyong/p/16557785.html

maven安装及使用：https://www.cnblogs.com/uncleyong/p/10743181.html

docker安装及使用：https://www.cnblogs.com/uncleyong/p/8894133.html

harbor安装以及使用：https://www.cnblogs.com/uncleyong/p/15469575.html

k8s安装：https://www.cnblogs.com/uncleyong/p/15499732.html（k8s操作：https://www.cnblogs.com/uncleyong/p/15499743.html）

jenkins搭建及devops自动化平台相关配置：https://www.cnblogs.com/uncleyong/p/16555667.html

allure报告：allure-commandline下载、安装、配置（linux或者docker），https://www.cnblogs.com/uncleyong/p/16726826.html

jenkins把自动化测试结果发送到钉钉群：https://www.cnblogs.com/uncleyong/p/16724590.html

06

pipeline设计

_____

pipeline常用功能：https://www.cnblogs.com/uncleyong/p/16705620.html

使用Blue Ocean设计pipeline脚本：https://www.cnblogs.com/uncleyong/p/16727971.html说明：由于资源不足，暂未加入sonarq

 

 

07

pipeline具体实现

 

_____

结合上面在Blue Ocean中设计的pipeline骨架，我们来完善并实现整个过程

parameters

可以选择要构建的分支 
parameters {
gitParameter branch: '', branchFilter: 'origin/(.*)', defaultValue: '', description: '构建的分支', name: 'BRANCH', quickFilterEnabled: false, selectedValue: 'NONE', sortMode: 'NONE', tagFilter: '*', type: 'GitParameterDefinition'
}
 

环境变量

environment {
  HARBOR_ADDRESS = "192.168.117.160"
  IMAGE_NAME = "gift"
  NAMESPACE = "gift"
}
 

agent

k8s作为构建环境 
  agent {
    kubernetes {
      cloud 'qzcsbj_kubernetes'
      yaml '''apiVersion: v1
kind: Pod
spec:
  containers:
    - image: 'registry.cn-chengdu.aliyuncs.com/qzcsbj6/jnlp:alpine'     
      name: jnlp
      imagePullPolicy: IfNotPresent
      args: [\'$(JENKINS_SECRET)\', \'$(JENKINS_NAME)\']
      volumeMounts:
        - mountPath: "/etc/localtime"
          name: "localtime"
          readOnly: false
    - image: "registry.cn-chengdu.aliyuncs.com/qzcsbj6/kubectl"
      imagePullPolicy: "IfNotPresent"
      name: "kubectl"
      tty: true
      command:
        - "cat"
      env:
        - name: "LANGUAGE"
          value: "en_US:en"
        - name: "LANG"
          value: "en_US.UTF-8"
      volumeMounts:
        - mountPath: "/etc/localtime"
          name: "localtime"
          readOnly: false
    - image: "registry.cn-chengdu.aliyuncs.com/qzcsbj6/docker"
      imagePullPolicy: "IfNotPresent"
      name: "docker"
      tty: true
      command:
        - "cat"
      env:
        - name: "LANGUAGE"
          value: "en_US:en"
        - name: "LANG"
          value: "en_US.UTF-8"
      volumeMounts:
        - mountPath: "/etc/localtime"
          name: "localtime"
          readOnly: false
        - mountPath: "/var/run/docker.sock"
          name: "dockersock"
          readOnly: false
    - image: "registry.cn-chengdu.aliyuncs.com/qzcsbj6/maven"
      imagePullPolicy: "IfNotPresent"
      name: "maven"
      tty: true
      command:
        - "cat"
      env:
        - name: "LANGUAGE"
          value: "en_US:en"
        - name: "LANG"
          value: "en_US.UTF-8"
      volumeMounts:
        - mountPath: "/etc/localtime"
          name: "localtime"
        - mountPath: "/root/.m2/"
          name: "m2dir"
          readOnly: false
    - image: "registry.cn-chengdu.aliyuncs.com/allure-commandline"
      imagePullPolicy: "IfNotPresent"
      name: "allure"
      tty: true
      command:
        - "cat"
      env:
        - name: "LANGUAGE"
          value: "en_US:en"
        - name: "LANG"
          value: "en_US.UTF-8"
      volumeMounts:
        - mountPath: "/etc/localtime"
          name: "localtime"
          readOnly: false
  restartPolicy: "Never"
  volumes:
    - name: "dockersock"
      hostPath:
        path: "/var/run/docker.sock"
    - name: "localtime"
      hostPath:
        path: "/usr/share/zoneinfo/Asia/Shanghai"
    - name: "m2dir"
      hostPath:
        path: "/opt/m2"
'''
    }
  }
 

pull project code

stage('pull project code') {
  parallel {
    stage('ui build') {
      when {
        expression {
          env.gitlabBranch == null
        }
      }
      steps {
        sh """
          echo '=================开始拉取项目代码'
        """

        git(url: 'git@192.168.117.180:qzcsbj/gift.git', branch: "${BRANCH}", credentialsId: 'qzcsbj_gitlab')
        script {
            COMMIT_ID = sh(returnStdout: true, script: "git log -n 1 --pretty=format:'%h'").trim()
            TAG = BUILD_TAG + '-' + COMMIT_ID
            println "Current branch is ${BRANCH}, Commit ID is ${COMMIT_ID}, Image TAG is ${TAG}"
        }           
      }
      post {
        failure {
          dingtalk (
            robot:'dd01',
            type:'MARKDOWN',
            atAll: true,
            title: "notice: 拉取项目代码失败",
            text: ["#### '${JOB_NAME}'项目代码拉取失败\n - 构建：第'${BUILD_NUMBER}'次\n - 状态：'${currentBuild.result}'\n - [查看本次构建详情](${BUILD_URL})"]
          )
        }
      }
    }
    stage('trigger build') {
      steps {
        sh """
          echo "待更新。。。"
        """
      }
    }
  }
}

 

mvn package

stage('mvn package') {
  steps {
    container(name: 'maven') {
      sh """
        echo '=================mvn开始打包'
        mvn clean package -Dmaven.test.skip=true
        echo '=================mvn打包完成'
      """
    }
  }
  post {
    failure {
      dingtalk (
        robot:'dd01',
        type:'MARKDOWN',
        atAll: true,
        title: "notice: mvn package失败",
        text: ["#### '${JOB_NAME}'项目mvn package失败\n - 构建：第'${BUILD_NUMBER}'次\n - 状态：'${currentBuild.result}'\n - [查看本次构建详情](${BUILD_URL})"]
      )
    }
  }
}
　　

build and push image

stage('build and push image') {
  environment {
    HARBOR_USER = credentials('qzcsbj_harbor')
  }
  steps {
    container(name: 'docker') {
      sh """
        echo '=================开始构建镜像'
        docker build -t ${HARBOR_ADDRESS}/${REGISTRY_DIR}/${IMAGE_NAME}:${TAG} .
        docker login -u ${HARBOR_USER_USR} -p ${HARBOR_USER_PSW} ${HARBOR_ADDRESS}
        docker push ${HARBOR_ADDRESS}/${REGISTRY_DIR}/${IMAGE_NAME}:${TAG}
        echo '=================镜像推送完成'
      """
    }
  }
  post {
    failure {
      dingtalk (
        robot:'dd01',
        type:'MARKDOWN',
        atAll: true,
        title: "notice: 构建或者推送镜像失败",
        text: ["#### '${JOB_NAME}'项目构建或者推送镜像失败\n - 构建：第'${BUILD_NUMBER}'次\n - 状态：'${currentBuild.result}'\n - [查看本次构建详情](${BUILD_URL})"]
      )
    }
  }
}
 

deploy to k8s

stage('deploy to k8s') {
  environment {
    MY_KUBECONFIG = credentials('qzcsbj_k8s')
  }
  steps {
    container(name: 'kubectl'){
      sh """
        echo '=================开始部署到k8s'
        /usr/local/bin/kubectl --kubeconfig $MY_KUBECONFIG set image deploy -l app=${IMAGE_NAME} ${IMAGE_NAME}=${HARBOR_ADDRESS}/${REGISTRY_DIR}/${IMAGE_NAME}:${TAG} -n $NAMESPACE
        echo '=================部署到k8s完成'
      """
    }
  }
  post {
    success {
      dingtalk (
        robot:'dd01',
        type:'MARKDOWN',
        atAll: true,
        title: "notice: 自动化部署到k8s完成",
        text: ["#### '${JOB_NAME}'项目自动化部署到k8s完成\n - 构建：第'${BUILD_NUMBER}'次\n - 状态：'${currentBuild.result}'\n - [查看本次构建详情](${BUILD_URL})"]
      )
    }
    failure {
      dingtalk (
        robot:'dd01',
        type:'MARKDOWN',
        atAll: true,
        title: "notice: 自动化部署到k8s失败",
        text: ["#### '${JOB_NAME}'项目自动化部署到k8s失败\n - 构建：第'${BUILD_NUMBER}'次\n - 状态：'${currentBuild.result}'\n - [查看本次构建详情](${BUILD_URL})"]
      )
    }
  }
}
 

pull autotest code

stage("pull autotest code"){
  steps{
    sh """
      echo '=================开始拉取自动化测试代码'
    """
    git(
      credentialsId: 'qzcsbj_gitlab',
      url: 'git@192.168.117.180:root/apiautotest.git'
    )
  }
  post {
    failure {
      dingtalk (
        robot:'dd01',
        type:'MARKDOWN',
        atAll: true,
        title: "notice: 拉取自动化测试代码失败",
        text: ["#### 拉取自动化测试代码失败\n - 构建：第'${BUILD_NUMBER}'次\n - 状态：'${currentBuild.result}'\n - [查看本次构建详情](${BUILD_URL})"]
      )
    }
  }
}
　　

run autotest

stage("run autotest"){
  steps{
    sh """
      mvn clean test -Dsurefire.suiteXmlFiles=testngXML/testng.xml
    """
  }
  post {
    success {
      dingtalk (
        robot:'dd01',
        type:'MARKDOWN',
        atAll: true,
        title: "notice: 自动化测试完成",
        text: ["#### '${JOB_NAME}'项目自动化测试完成\n - 构建：第'${BUILD_NUMBER}'次\n - 状态：'${currentBuild.result}'\n - [查看本次构建详情](${BUILD_URL}console)"]
      )
    }
    failure {
      dingtalk (
        robot:'dd01',
        type:'MARKDOWN',
        atAll: true,
        title: "notice: 自动化测试运行出错",
        text: ["#### '${JOB_NAME}'项目自动化测试运行出错\n - 构建：第'${BUILD_NUMBER}'次\n - 状态：'${currentBuild.result}'\n - [查看本次构建详情](${BUILD_URL})"]
      )
    }
  }
}
　　

allure report

stage("allure report"){
  steps{
    sh """
      echo '=================开始生成测试报告'
    """
    allure(
      includeProperties: false,
      jdk: '',
      results: [[path: 'target/allure-results']]
    )
  }
  post {
    success {
      dingtalk (
        robot:'dd01',
        type:'MARKDOWN',
        atAll: true,
        title: "notice: 自动化测试报告已生成，全部通过",
        text: ["#### '${JOB_NAME}'项目自动化测试报告已生成，全部通过\n - 构建：第'${BUILD_NUMBER}'次\n - 状态：'${currentBuild.result}'\n - [查看本次测试报告](${BUILD_URL}allure)"]
      )
    }
    unstable {
      dingtalk (
        robot:'dd01',
        type:'MARKDOWN',
        atAll: true,
        title: "notice: 自动化测试报告已生成，有未通过的",
        text: ["#### '${JOB_NAME}'项目自动化测试报告已生成，有未通过的\n - 构建：第'${BUILD_NUMBER}'次\n - 状态：'${currentBuild.result}'\n - [查看本次测试报告](${BUILD_URL}allure)"]
      )
    }
    failure {
      dingtalk (
        robot:'dd01',
        type:'MARKDOWN',
        atAll: true,
        title: "notice: 自动化测试报告生成出错",
        text: ["#### '${JOB_NAME}'项目自动化测试报告生成出错\n - 构建：第'${BUILD_NUMBER}'次\n - 状态：'${currentBuild.result}'\n - [查看本次构建详情](${BUILD_URL})"]
      )
    }
  }
}
 

08

构建及效果展示

 

_____

pipeline脚本可以放在流水线项目的脚本框中

 

 

 

也可以放gitlab上，选择SCM

 

 

 

 

 

 

  
pipeline脚本放到Jenkinsfile中，Jenkinsfile在gift项目master分支下

 

 


第一次构建，点击“Build Now”，会失败，因为我们使用了参数化构建，这次构建获取不到参数值

 

 

 

刷新页面，就可以看到参数化构建了，就可以选择要构建的分支

 

 


部署到k8s的通知

 

 

消息

 

 

 

运行自动化测试的通知

 

 

消息

 

 

 

生成allure测试报告的通知

 

 

消息

 

 

测试报告的通知可以优化，直接链接到allure测试报告

 

 


测试报告

 

 

posted @   AlamZ  阅读(1833)  评论(1)  编辑  收藏  举报
相关博文：
· 基于docker部署实现接口自动化持续集成
· 自动化测试 RF+Jenkins+git
· 企业级持续集成实战系列（01）：持续集成流程设计
· 【自动化持续集成必备基础】pipeline常用功能
· Kubernetes配合Jenkins实现轻量自动持续集成
阅读排行：
· 全程不用写代码，我用AI程序员写了一个飞机大战
· DeepSeek 开源周回顾「GitHub 热点速览」
· MongoDB 8.0这个新功能碉堡了，比商业数据库还牛
· 记一次.NET内存居高不下排查解决与启示
· 白话解读 Dapr 1.15：你的「微服务管家」又秀新绝活了
点击右上角即可分享
微信分享提示