基于Jenkins构建微服务项目发布失败回滚

基于 Jenkins 构建微服务项目发布失败回滚

1 微服务项目发布失败回滚

1.1 安装插件 Active Choices(动态生成选项)

1.2 编写获取 harbor 仓库镜像列表的脚本

[root@k8s-master1 ms_jenkins]# cat get_tags.sh
#!/bin/bash
​
# Harbor的用户名
USER="admin"
# Harbor的密码
PASS="Harbor12345"
# Harbor的地址
HURL="http://172.16.1.61"
# Harbor的项目名
PROJECT_NAME=$1
# 选择镜像的名
SERVICE_NAME=$2
SERVICE_NAME="${SERVICE_NAME%-*}"
​
ttoken=$(curl -iksL -X GET -u $USER:$PASS $HURL/service/token?account=${USER}\&service=harbor-registry\&scope=repository:${PROJECT_NAME}/${SERVICE_NAME}:pull |grep "token" |awk -F '"' '{print $4}')
​
tlist=$(curl -ksL -X GET -H "Content-Type: application/json" -H "Authorization: Bearer $ttoken" ${HURL}/v2/${PROJECT_NAME}/${SERVICE_NAME}/tags/list |awk -F '[' '{print $2}' |awk -F ']' '{print $1}' |sed 's/"//g')
​
echo $tlist |sed 's/,/\n/g' |sort -nr |head -n 5
# 输出结果
# 5
# 4
# 3
# 2
# 1

1.3 将 get_tags.sh 脚本拷贝到 jenkins 家目录下

[root@k8s-master1 ms_jenkins]# kubectl get pod -n default | grep jenkins
jenkins-687d65dff5-7rh9v                  1/1     Running   5         3d1h
​
[root@k8s-master1 ms_jenkins]# kubectl cp get_tags.sh jenkins-687d65dff5-7rh9v:/var/jenkins_home/

1.4 在 jenkins 上给要回滚的项目创建一个 Job

1.5 pipeline 脚本

#!/usr/bin/env groovy
// 所需插件: Git Parameter/Git/Pipeline/Config File Provider/kubernetes/Extended Choice Parameter/Active Choices
​
// harbor仓库地址
def registry = "172.16.1.61"
// harbor仓库项目名称
def project = "microservice"
// jenkins agent 镜像
jenkins_agent = "172.16.1.61/library/jenkins-slave-jdk:1.8"
​
// gitlab仓库地址
def git_url = "http://172.16.1.62:9999/dev/microservice.git"
​
// jenkins存储的认证信息
def git_auth = "e80eb4fe-4564-48f5-9ef4-a75b49c1c079"
def k8s_auth = "5c5ae334-12dd-4a6a-8a47-1e2a1dbed983"
​
pipeline {
  agent {
    kubernetes {
        label "jenkins-agent"
        yaml """
apiVersion: v1
kind: Pod
metadata:
  name: jenkins-agent
spec:
  containers:
  - name: jnlp
    image: "${jenkins_agent}"
    imagePullPolicy: Always
    volumeMounts:
      - name: docker-cmd
        mountPath: /usr/bin/docker
      - name: docker-sock
        mountPath: /var/run/docker.sock
  volumes:
    - name: docker-cmd
      hostPath:
        path: /usr/bin/docker
    - name: docker-sock
      hostPath:
        path: /var/run/docker.sock
"""
   }
  }
​
    parameters {
      choice choices: ['eureka-service', 'gateway-service', 'portal-service', 'product-service', 'order-service', 'stock-service'], \
        description: '请选择要回滚的应用', \
        name: 'Service'
​
      reactiveChoice choiceType: 'PT_SINGLE_SELECT', description: '请选择回滚应用的镜像版本', \
        filterLength: 1, filterable: false, name: 'Tag', \
        randomName: 'choice-parameter-2431088085046', referencedParameters: 'Service', \
        script: groovyScript(fallbackScript: [classpath: [], oldScript: '', sandbox: true, script: ''], \
        script: [classpath: [], oldScript: '', sandbox: true, \
        script: '''cmd = "/bin/bash /var/jenkins_home/get_tags.sh microservice ${Service}"
                   tags_list = cmd.execute().text.tokenize()
                   return tags_list''' \
       ])
   }
​
    stages {
        stage('执行回滚操作') {
            steps {
                checkout scmGit(branches: [[name: "origin/master"]], extensions: [], \
                  userRemoteConfigs: [[credentialsId: "${git_auth}", url: "${git_url}"]])
                
                configFileProvider([configFile(fileId: "${k8s_auth}", targetLocation: 'admin.kubeconfig')]) {
                  sh """
                    service=${Service}
                    service_name=\${service%-*}
                    rollback_image=${registry}/${project}/\${service_name}:${Tag}
                    sed -i -r "s#(image: )(.*)#\\1\${rollback_image}#" k8s/\${service_name}.yaml
                    kubectl apply -f k8s/\${service_name}.yaml --kubeconfig admin.kubeconfig
                  """
               }
           }
       }
   }
}

1.6 将 pipeline 脚本复制到 pipeline script 文本框中

1.7 立即构建 job

第一次构建会失败并生成参数化构建选项。

1.8 参数化构建 job

1 由于使用了从 harbor 仓库获取镜像列表的脚本，jenkins 出于安全考虑禁止了该脚本运行，Tag 标签内容无法显示，需要信任该脚本。

2 系统管理 -> 安全

3 点击 Approve

4 根据选择要回滚的应用，Tag 会自动显示该应用镜像所有的标签

1.9 回滚测试

以 product-service 应用为例进行镜像版本的回滚操作。

1 查看 product-service 应用 pod 现在使用的像版本

[root@k8s-master1 ms_jenkins]# kubectl get pod -n ms | grep product
product-78d4fd9dcd-rd4wr   1/1     Running   0         5h13m
​
[root@k8s-master1 ms_jenkins]# kubectl describe pod/product-78d4fd9dcd-rd4wr -n ms | grep -i image
Image:          172.16.1.61/microservice/product:3

2 将 product-service 应用镜像版本从 product:3 回滚到 product:2

3 Console Output

......

4 查看流水线步骤

5 product-service 应用会在新的 pod running 后 Terminating 旧的 pod

6 检查 product-service 应用 pod 现在使用的像版本

[root@k8s-master1 ms_jenkins]# kubectl get pod -n ms | grep product
product-7dd5d6bbcc-57dx6   1/1     Running   0         21m
​
[root@k8s-master1 ms_jenkins]# kubectl describe pod/product-7dd5d6bbcc-57dx6 -n ms | grep -i image
   Image:          172.16.1.61/microservice/product:2