Jenkins和Gitlab CI/CD自动更新k8s中pod使用的镜像说明
Jenkins
使用Jenkins的话，完成的工作主要有如下步骤：
1.从Gogs或Gitlab仓库上拉取代码
2.使用Maven编译代码，打包成jar文件
3.根据jar文件使用相对应的Dockerfile文件制作成Docker镜像
4.把Docker镜像推送到Nexus上的Docker仓库(或者Harbor仓库)
5.运行shell脚本，给k8s的master主机上执行更新pod的脚本命令 (这一步不想自动实现的话可以采取手动操作)
前四步的操作：
地址：https://www.cnblogs.com/sanduzxcvbnm/p/11803368.html
https://www.cnblogs.com/sanduzxcvbnm/p/11800008.html
https://www.cnblogs.com/sanduzxcvbnm/p/11799583.html
https://www.cnblogs.com/sanduzxcvbnm/p/11797660.html
第5步的操作说明：
k8s使用的是kuboard面板提供的有一个CI/CD集成功能，主要使用的就是这个
官方网址：https://kuboard.cn/guide/cicd/
直接在 shell 脚本中调用 Kuboard/Kubernetes API，更新对应容器镜像的标签。完成此接口调用后，Kubernetes 将自动完成滚动更新。
界面中提供的脚本里，容器的版本是当前 Kubernetes 中已部署的版本，通常在 Jenkins/GitlabRunner 之类的工具里，需要将该脚本的镜像标签参数化。
Gitlab Runner
使用Jenkins的话，完成的工作主要有如下步骤：
1.代码提交到Gitlab仓库后根据检测到的.gitlab-ci.yml文件自动执行CI/CD操作
2.CI/CD操作
2.1使用Maven编译代码，打包成jar文件
2.2根据jar文件使用相对应的Dockerfile文件制作成Docker镜像
2.3把Docker镜像推送到Nexus上的Docker仓库(或者Harbor仓库) (这一步使用的变量可以在Gitlab中进行设置)
2.4运行shell脚本，给k8s的master主机上执行更新pod的脚本命令 (还有另一种办法，详看下面) (这一步不想自动实现的话可以采取手动操作)
.gitlab-ci.yml文件内容示例：
stages:
- test
- build
- release
- review
- deploy
test:
stage: test
tags:
- test
script:
- sleep 3
- echo "We did it! Something else runs in parallel!"
compile:
image: maven:3.5-jdk-8-alpine
tags:
- maven
stage: build
only:
- dev
script:
- mvn clean package -Dmaven.test.skip=true
artifacts:
paths:
- target/*.jar
- target/lib
image_build:
stage: release
image: docker:latest
tags:
- docker
services:
- docker:dind
variables:
TAG: v0.9 # 镜像版本，需要想办法设置灵活
script:
- docker build -t "${CI_REGISTRY_IMAGE}:$TAG" .
- docker rm -f test || true
- docker login -u "${CI_REGISTRY_USER}" -p "${CI_REGISTRY_PASSWORD}" "${CI_REGISTRY}"
- docker tag "${CI_REGISTRY_IMAGE}:$TAG" "${CI_REGISTRY}/${CI_REGISTRY_IMAGE}:$TAG"
- docker push "${CI_REGISTRY}/${CI_REGISTRY_IMAGE}:$TAG"
only:
- dev
pod_update:
stage: deploy
tags:
- k8s
script:
- echo "=============== 开始执行更新pod使用的镜像任务 ==============="
- pwd
- chmod a+x ./scripts/build/update.sh
# 这一步执行报错：不是找不到文件就是找不到命令，原因采用的注册runner时的docker镜像，里面没有curl命令，解决办法：在这一步找一个含有curl命令的镜像即可
# - ./scripts/build/update.sh
- echo "=============== 更新任务执行结束 ============================"
默认在注册runner的时候需要填写一个基础的镜像，只要使用执行器为docker类型的runner所有的操作运行都会在容器中运行。 如果全局指定了images则所有作业使用此image创建容器并在其中运行。 全局未指定image，再次查看job中是否有指定，如果有此job按照指定镜像创建容器并运行，没有则使用注册runner时指定的默认镜像。
update.sh文件内容跟Jenkins中的第五步操作一样
Gitlab Runner上跟K8S上更新pod的另一个操作：
在.gitlab-ci.yml文件安装kubectl工具，然后使用这个工具客户端跟k8s进行通信，从而执行更新pod的操作
安装kubectl工具地址：https://www.cnblogs.com/sanduzxcvbnm/p/13865238.html
示例内容如下：
image:
name: golang:1.10.3-stretch
entrypoint: ["/bin/sh", "-c"]
# The problem is that to be able to use go get, one needs to put
# the repository in the $GOPATH. So for example if your gitlab domain
# is mydomainperso.com, and that your repository is repos/projectname, and
# the default GOPATH being /go, then you'd need to have your
# repository in /go/src/mydomainperso.com/repos/projectname
# Thus, making a symbolic link corrects this.
before_script:
- mkdir -p "/go/src/git.qikqiak.com/${CI_PROJECT_NAMESPACE}"
- ln -sf "${CI_PROJECT_DIR}" "/go/src/git.qikqiak.com/${CI_PROJECT_PATH}"
- cd "/go/src/git.qikqiak.com/${CI_PROJECT_PATH}/"
stages:
- test
- build
- release
- review
- deploy
test:
stage: test
script:
- make test
test2:
stage: test
script:
- sleep 3
- echo "We did it! Something else runs in parallel!"
compile:
stage: build
script:
# Add here all the dependencies, or use glide/govendor/...
# to get them automatically.
- make build
artifacts:
paths:
- app
image_build:
stage: release
image: docker:latest
variables:
DOCKER_DRIVER: overlay
DOCKER_HOST: tcp://localhost:2375
services:
- name: docker:17.03-dind
command: ["--insecure-registry=registry.qikqiak.com"]
script:
- docker info
- docker login -u "${CI_REGISTRY_USER}" -p "${CI_REGISTRY_PASSWORD}" registry.qikqiak.com
- docker build -t "${CI_REGISTRY_IMAGE}:latest" .
- docker tag "${CI_REGISTRY_IMAGE}:latest" "${CI_REGISTRY_IMAGE}:${CI_COMMIT_REF_NAME}"
- test ! -z "${CI_COMMIT_TAG}" && docker push "${CI_REGISTRY_IMAGE}:latest"
- docker push "${CI_REGISTRY_IMAGE}:${CI_COMMIT_REF_NAME}"
deploy_review:
image: cnych/kubectl
stage: review
only:
- branches
except:
- tags
environment:
name: dev
url: https://dev-gitlab-k8s-demo.qikqiak.com
on_stop: stop_review
script:
- kubectl version
- cd manifests/
- sed -i "s/__CI_ENVIRONMENT_SLUG__/${CI_ENVIRONMENT_SLUG}/" deployment.yaml ingress.yaml service.yaml
- sed -i "s/__VERSION__/${CI_COMMIT_REF_NAME}/" deployment.yaml ingress.yaml service.yaml
- |
if kubectl apply -f deployment.yaml | grep -q unchanged; then
echo "=> Patching deployment to force image update."
kubectl patch -f deployment.yaml -p "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"ci-last-updated\":\"$(date +'%s')\"}}}}}"
else
echo "=> Deployment apply has changed the object, no need to force image update."
fi
- kubectl apply -f service.yaml || true
- kubectl apply -f ingress.yaml
- kubectl rollout status -f deployment.yaml
- kubectl get all,ing -l ref=${CI_ENVIRONMENT_SLUG}
stop_review:
image: cnych/kubectl
stage: review
variables:
GIT_STRATEGY: none
when: manual
only:
- branches
except:
- master
- tags
environment:
name: dev
action: stop
script:
- kubectl version
- kubectl delete ing -l ref=${CI_ENVIRONMENT_SLUG}
- kubectl delete all -l ref=${CI_ENVIRONMENT_SLUG}
deploy_live:
image: cnych/kubectl
stage: deploy
environment:
name: live
url: https://live-gitlab-k8s-demo.qikqiak.com
only:
- tags
when: manual
script:
- kubectl version
- cd manifests/
- sed -i "s/__CI_ENVIRONMENT_SLUG__/${CI_ENVIRONMENT_SLUG}/" deployment.yaml ingress.yaml service.yaml
- sed -i "s/__VERSION__/${CI_COMMIT_REF_NAME}/" deployment.yaml ingress.yaml service.yaml
- kubectl apply -f deployment.yaml
- kubectl apply -f service.yaml
- kubectl apply -f ingress.yaml
- kubectl rollout status -f deployment.yaml
- kubectl get all,ing -l ref=${CI_ENVIRONMENT_SLUG}
