golang程序通过docker打包到harbor的方式进行jenkins自动化发布
需求描述：
公司内部的git仓库 golang 服务端代码需要以docker打包的方式发布到外网多台服务器中，作为api接口服务启动，然后nginx反向代理到这几台机器中
如果不以docker的方式启动直接编译发布启动也是可以的，以docker启动方式主要是为后续上k8s等类似的平台做技术铺垫
大概的思路：
1.jenkins拉取git代码通过rsync推送到远端服务器中(因为是golang代码，依赖的go模块在公司内部网络git，需要在公司内部编译生成可执行的文件，推送到外网进行docker打包)
2.在远端搭建docker的仓库harbor服务器
3.通过jenkins的方式调取远端服务器中的脚本进行 doker 的打包并推送到harbor中，然后拉取harbor中的镜像启动服务
架构图
1.在远端服务器中安装docker，在其中一台服务器中安装harbor作为镜像服务器
# 安装docker
yum install -y yum-utils
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum install -y docker-ce
环境准备
mkdir -p /data/www/vhosts/docker/storage/logs
chown -R apache.users /data/www/vhosts/docker/
2.修改docker配置(每个服务端的ip端最好不同)
mkdir /etc/docker
# vim /etc/docker/daemon.json
{ "graph": "/data/docker", "storage-driver": "overlay2", "insecure-registries": ["registry.access.redhat.com","quay.io","172.30.0.122:1800"], "registry-mirrors": ["https://q2gr04ke.mirror.aliyuncs.com"], "bip": "10.30.136.1/24", "exec-opts": ["native.cgroupdriver=systemd"], "live-restore": true }
"bip": "10.30.162.1/24" 说明：
最好不要和公司内部或者vpc的网段冲突
服务器IP：1.2.17.136/172.30.0.70
对应
10.30.136.1/24
# 启动docker
systemctl start docker
systemctl enable docker
# 安装 harbor 仓库
参考：https://www.cnblogs.com/reblue520/p/13615972.html
修改harbor仓库的地址
hostname: 172.30.0.122
port: 1800
harbor_admin_password: pass
# 远程需要部署docker 安装包api接口的服务器上 通过apache用户登录docker harbor
# 生成config.json 这个秘钥文件，这样就可以在远程服务器中执行 pull push 命令而不需要输入账号密码直接执行了
# more /data/www/.docker/config.json { "auths": { "172.30.0.122:1800": { "auth": "efadYXJib3IyMDIxYWd2W4=" } } }
3.将apache用户添加可以运行docker的权限
sudo usermod -aG docker apache
4.编写远程脚本脚本
mkdir /usr/local/worksh/jeninks_task/
# cat /usr/local/worksh/jeninks_task/vidiar_search_docker_build_restart.sh
#!/bin/bash ENV=$1 TAG=$2 # 复制环境变量 cd /data/www/vhosts/vidair-search.chinasoft.com/httpdocs cp .env.$ENV .env # 执行权限 chmod 755 vidair-search.exe # 生成docker image cd /data/www/vhosts/vidair-search.chinasoft.com/httpdocs docker build -t 172.30.0.122:1800/todd/vidair-search:$TAG . # 推送到docker仓库 cd /data/www/vhosts/vidair-search.chinasoft.com/httpdocs docker push 172.30.0.122:1800/todd/vidair-search:$TAG # 通知远程机器重新拉取image并且启动container docker pull 172.30.0.122:1800/todd/vidair-search:${TAG} docker stop vidair-search docker rm -f vidair-search docker container run -p 3012:3001 -it -d -v /data/www/vhosts/docker/storage/logs:/app/storage/logs -v /nasplatform:/nasplatform -v /nasuser:/nasuser -v /tmpdata:/tmpdata --name=vidair-search 172.30.0.122:1800/todd/vidair-search:${TAG}
# 添加可执行权限
chmod +x /usr/local/worksh/jeninks_task/vidiar_search_docker_build_restart.sh
chown -R apache.users /usr/local/worksh/jeninks_task
# Dockerfile
# more Dockerfile
From centos RUN mkdir -p /app/storage/logs WORKDIR /app COPY vidair-search.exe /app/vidair-search.exe COPY .env /app/.env EXPOSE 3001 CMD ["/app/vidair-search.exe"]
# 修改 Dockerfile 这样可以在不能启动 docker 容器成功的情况下进入容器中排查问题
# more Dockerfile
From centos RUN mkdir -p /app/storage/logs WORKDIR /app COPY vidair-search.exe /app/vidair-search.exe COPY .env /app/.env EXPOSE 3001 # CMD ["/app/vidair-search.exe"] CMD ["/bin/bash","-c","tail -f /dev/null"]
5.配置jenkins发布环境
git仓库相关配置
jekins shell脚本
#!/bin/bash # 此脚本功能为根据构建时选择的参数，同步 /data/www/vhosts/vidair-search.chinasoft.com.prod 下的文件同步到远程中转机器 # 2021.05.21 初始化脚本 #非apache用户运行脚本，则退出 if [ `whoami` != "apache" ];then echo "only apache can run me" exit 1 fi echo "xxx" ## 1.定义变量 dir_name=bak.$(date +%Y-%m-%d-%H-%M-%S) project_dir=/data/www/vhosts/vidair-search.chinasoft.com.prod ## 2.备份代码函数 function func_project_backup(){ cp -a $project_dir/ /data/data_backup/vidair-search.chinasoft.com.prod_$dir_name } #func_project_backup ## 3.判断代码发布目录变量是否为空 if [ ! $project_dir ];then echo "$project_dir IS NULL ,shell exit!!!!" exit 1 fi ## 2.判断同步状态 function func_rsync_status(){ if [[ $? == 0 || $? == 23 ]];then rsync_edit=1 else rsync_edit=0 echo "`date` 同步到本地目标失败! " exit 1 fi } ## 4.同步到本地待发路径 function func_rsync_project_local(){ echo "xxxxxxxxxxxxxx同步待发目录开始xxxxxxxxxxxxxxxxxx" cd $WORKSPACE /usr/local/bin/rsync -vau -progress --delete --exclude='.git/' --exclude='.gitignore' --exclude='*.log' $WORKSPACE/ $project_dir/httpdocs/ func_rsync_status echo "xxxxxxxxxxxxxx同步待发目录完成xxxxxxxxxxxxxxxxxx" } func_rsync_project_local # 执行编译(必须在公司内部编译，因为编译打包依赖内网git仓库) cd $project_dir/httpdocs/ && chmod +x ./init.production.sh && ./init.production.sh production chown -R apache.users $project_dir/ sleep 1 ## 5.推送代码到远程中转机并发布(发布到线上) echo "------------------------------------ rsync start prod -----------------------------------------" chown -R apache.users $project_dir/ sleep 1 /bin/bash /usr/local/worksh/jeninks_task/online_video.vidair-search.chinasoft.com.prod.sh echo "------------------------------------ rsync done prod -----------------------------------------" ## 7.通过插件执行远程服务器中的脚本生成docker镜像，然后推送到仓库中，最后推送到目标服务器中，并启动
涉及的 init.production.sh 脚本内容
#!/bin/bash -l ENV=$1 # 复制环境变量 cp .env.$ENV .env # 处理依赖 go mod tidy go mod vendor # 构建启动程序 GOARCH=amd64 GOOS=linux go build -ldflags "-s -w" -o vidair-search.exe main.go # 执行权限 chmod 755 vidair-search.exe
发布代码脚本
# cat /usr/local/worksh/jeninks_task/online_video.vidair-search.chinasoft.com.prod.sh
#!/bin/bash ############################################# ## 设置变量和GET请求过来的变量 ## GET请求传过来的文件所在目录,目录路径写全路径了 #dir=$1 passfile="/data/www/.rsync/pass.oneline-video.vidair-search.chinasoft.com.prod" # 非apache用户运行脚本，则退出 if [ `whoami` != "apache" ];then echo " only apache can run me" exit 1 fi # 判断同步状态 function func_rsync_status(){ if [[ $? == 0 || $? == 23 ]];then rsync_edit=1 else rsync_edit=0 echo "`date` 同步到目标失败! " exit 1 fi } # 判断目录是否为空函数 function func_is_empty_dir(){ return `ls -A $1|wc -w` } # 代码发目录 project_dir="/data/www/vhosts/vidair-search.chinasoft.com.prod/" # 判断待发目录是否为空，为空则退出 if func_is_empty_dir $project_dir then echo " $project_dir is empty , exit!!!!" exit 1 else echo " $project_dir 可以发布" fi ## 设置变量，目标服务器 server_ip_list="1.1.1.1 1.1.1.2" # src directory src_directory="vidair-search.chinasoft.com.prod" # dst directory dst_directory="vidair-search.chinasoft.com" exclude_list="--exclude=.svn --exclude=.git --exclude=.gitignore --exclude=*.log --exclude=.gitattributes --exclude=.env" function vidair_rsyn_prod() { # rsync ip_list for ip in ${server_ip_list} do echo "####################rsync ${ip} start################################" rsync -zavP --bwlimit=1000 ${exclude_list} --password-file=${passfile} /data/www/vhosts/${src_directory}/ apache@${ip}::apache/data/www/vhosts/${dst_directory}/ func_rsync_status echo "################### rsync ${ip} end #######################" done } vidair_rsyn_prod exit 0
编译时的发布参数
# 通过 docker-compose 启动docker容器
# more /usr/local/worksh/jeninks_task/vidiar_search_docker_build_restart.sh
#!/bin/bash ENV=$1 TAG=$2 # 复制环境变量 cd /data/www/vhosts/vidair-search.chinasoft.com/httpdocs cp .env.$ENV .env # 执行权限 chmod 755 vidair-search.exe # 生成docker image cd /data/www/vhosts/vidair-search.chinasoft.com/httpdocs docker build -t 172.30.0.122:1800/vidair/search-service:$TAG . docker build -t 172.30.0.122:1800/vidair/search-service-worker:$TAG . # 推送到docker仓库 cd /data/www/vhosts/vidair-search.chinasoft.com/httpdocs docker push 172.30.0.122:1800/vidair/search-service:$TAG docker push 172.30.0.122:1800/vidair/search-service-worker:$TAG # 通知远程机器重新拉取image并且启动container #docker pull 172.30.0.122:1800/vidair/vidair-search:${TAG} #docker stop vidair-search #docker rm -f vidair-search #docker container run -p 3012:3001 -it -d -v /data/www/vhosts/docker/storage/logs:/app/storage/logs -v /nasplatform:/nasplatform -v /nasuser:/nasuser -v /tmpdata:/tmpdata --name=vidair-search 172.30.0.122:1800/vidair/vidair-search:${TAG} #docker-compose -f /data/www/vhosts/vidair-search.chinasoft.com/httpdocs/docker-compose.production.yml up --scale worker=3 -d TAG=${TAG} docker-compose -f /data/www/vhosts/vidair-search.chinasoft.com/httpdocs/docker-compose.production.yml up --scale worker=3 -d
# compose文件
# more /data/www/vhosts/vidair-search.chinasoft.com/httpdocs/docker-compose.production.yml
version: '3' services: web: image: 172.30.0.122:1800/vidair/search-service:${TAG} container_name: vidair-search-container command: ["/app/vidair-search.exe"] ports: - "3012:3001" volumes: - "./storage/logs:/app/storage/logs" - "/nasplatform:/nasplatform" - "/nasuser:/nasuser" - "/tmpdata:/tmpdata" worker: image: 172.30.0.122:1800/vidair/search-service-worker:${TAG} command: ["/app/vidair-search.exe","worker"] volumes: - "./storage/logs:/app/storage/logs" - "/nasplatform:/nasplatform" - "/nasuser:/nasuser" - "/tmpdata:/tmpdata"
容器情况情况
# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES f554731ffe77 172.30.0.122:1800/vidair/search-service-worker:1.0.0 "/app/vidair-search.…" 36 minutes ago Up 36 minutes 3001/tcp httpdocs_worker_3 512c5ba84297 172.30.0.122:1800/vidair/search-service:1.0.0 "/app/vidair-search.…" 36 minutes ago Up 36 minutes 0.0.0.0:3012->3001/tcp, :::3012->3001/tcp vidair-search-container 0921a7e16cfc 172.30.0.122:1800/vidair/search-service-worker:1.0.0 "/app/vidair-search.…" 36 minutes ago Up 36 minutes 3001/tcp httpdocs_worker_2 9e353c484e80 172.30.0.122:1800/vidair/search-service-worker:1.0.0 "/app/vidair-search.…" 36 minutes ago Up 36 minutes 3001/tcp httpdocs_worker_1
