使用docker安装极狐gitlab+runner
1. 环境信息
1.1 主机信息
| 主机 | 用途 |
|---|---|
| 10.10.10.60 | 极狐gitlab |
| 10.10.10.61 | runner |
- 域名 gitlab.example.com，指向 10.10.10.60，这里使用设置本机 hosts 的方式解析，如果有条件的话使用本地 dns 服务器设置解析
1.2 配置信息
[root@localhost ~]# cat /etc/centos-release
CentOS Linux release 7.9.2009 (Core)
[root@localhost ~]#
[root@localhost ~]# uname -r
3.10.0-1160.el7.x86_64
[root@localhost ~]#
[root@localhost ~]# free -m
total used free shared buff/cache available
Mem: 3770 272 3284 11 213 3271
Swap: 3967 0 3967
[root@localhost ~]#
[root@localhost ~]# lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 2
On-line CPU(s) list: 0,1
Thread(s) per core: 1
Core(s) per socket: 2
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 141
Model name: 11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz
Stepping: 1
CPU MHz: 2304.002
BogoMIPS: 4608.00
Hypervisor vendor: VMware
Virtualization type: full
L1d cache: 48K
L1i cache: 32K
L2 cache: 1280K
L3 cache: 24576K
NUMA node0 CPU(s): 0,1
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology tsc_reliable nonstop_tsc eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 arat avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq movdiri movdir64b md_clear spec_ctrl intel_stibp flush_l1d arch_capabilities
[root@localhost ~]#
1.3 环境初始化
echo "设置时钟同步"
yum install -y chrony
systemctl start chronyd.service
systemctl enable chronyd.service
echo "关闭防火墙"
systemctl stop firewalld
systemctl disable firewalld
echo "关闭 selinux"
setenforce 0
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
getenforce
echo "优化 ssh 登录速度"
sed -i 's/#UseDNS yes/UseDNS no/g' /etc/ssh/sshd_config
systemctl restart sshd
echo "安装基础软件 lrzsz, epel-release"
yum install epel-release lrzsz -y
1.4 升级内核
rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
# 安装ELRepo
rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm
# 载入elrepo-kernel元数据
yum --disablerepo=\* --enablerepo=elrepo-kernel repolist
# 查看可用的rpm包
yum --disablerepo=\* --enablerepo=elrepo-kernel list kernel*
# 安装长期支持版本的kernel
yum --disablerepo=\* --enablerepo=elrepo-kernel install -y kernel-lt.x86_64
# 删除旧版本工具包
yum remove kernel-tools-libs.x86_64 kernel-tools.x86_64 -y
# 安装新版本工具包
yum --disablerepo=\* --enablerepo=elrepo-kernel install -y kernel-lt-tools.x86_64
#查看默认启动顺序
awk -F\' '$1=="menuentry " {print $2}' /etc/grub2.cfg
#默认启动的顺序是从0开始，新内核是从头插入（目前位置在0，而4.4.4的是在1），所以需要选择0。
grub2-set-default 0
reboot
- 当前最新稳定内核版本
CentOS Linux (5.4.180-1.el7.elrepo.x86_64) 7 (Core)
1.5 安装 docker
参考：Get Docker | Docker Documentation 或者
echo "
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1" >> /etc/sysctl.conf
sysctl -p
curl https://releases.rancher.com/install-docker/20.10.sh | sh
systemctl enable docker
设置国内源
vi /etc/docker/daemon.json
{
"registry-mirrors": ["https://hccwwfjl.mirror.aliyuncs.com"]
}
systemctl restart docker
- 其他源：http://hub-mirror.c.163.com，建议用这个 163 的
2. 安装极狐Gitlab
参考官方文档：GitLab Docker images | GitLab
2.1 开始安装
首先创建目录
mkdir -p /srv/gitlab
export GITLAB_HOME=/srv/gitlab
启动命令
docker run -d \
--hostname gitlab.example.com \
--env GITLAB_OMNIBUS_CONFIG="external_url 'http://gitlab.example.com/';gitlab_rails['initial_root_password'] = 'yourpassword';gitlab_rails['initial_shared_runners_registration_token'] = 'yourtoken'" \
--publish 443:443 --publish 80:80 --publish 2222:22 \
--name gitlab \
--restart always \
--volume $GITLAB_HOME/config:/etc/gitlab \
--volume $GITLAB_HOME/logs:/var/log/gitlab \
--volume $GITLAB_HOME/data:/var/opt/gitlab \
--shm-size 256m \
registry.gitlab.cn/omnibus/gitlab-jh:15.1.3
- 指定容器 22 端口映射到 2222 端口，避免和宿主机 ssh 端口冲突
- initial_root_password 指定初始化管理员密码
- initial_shared_runners_registration_token 指定 runner 连接 token
| 宿主机目录 | 容器目录 | 用途 |
|---|---|---|
$GITLAB_HOME/data |
/var/opt/gitlab |
程序数据 |
$GITLAB_HOME/logs |
/var/log/gitlab |
程序日志 |
$GITLAB_HOME/config |
/etc/gitlab |
极狐gitlab配置文件 |
2.2 配置方法
可以进入容器编辑 /etc/gitlab/gitlab.rb
docker exec -it gitlab /bin/bash
或者直接编辑文件
docker exec -it gitlab editor /etc/gitlab/gitlab.rb
编辑完成后需要重启容器生效
docker restart gitlab
2.3 访问极狐gitlab
访问地址：https://gitlab.example.com
- 需要本机设置 gitlab.example.com 的 hosts 记录
3. 安装 Runner
3.1 创建配置目录
mkdir -p /srv/gitlab-runner/config
3.2 启动 runner
docker run -d --name gitlab-runner --restart always \
--env TZ=Asia/Shanghai \
--add-host=gitlab.example.com:10.10.10.60 \
-v /srv/gitlab-runner/config:/etc/gitlab-runner \
-v /var/run/docker.sock:/var/run/docker.sock \
gitlab/gitlab-runner:v15.1.0
- mac 本机配置了 hosts，那么 hosts 配置对 docker 容器里面的所有容器都适用，但是在 linux 的时候就不适用了，所以这里使用 --add-host 添加容器里的 hosts 解析
3.3 注册 runner
docker run --rm -it --add-host=gitlab.example.com:10.10.10.60 -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner:v15.1.0 register
查看配置文件
$ cd /srv/gitlab-runner/config
$ cat config.toml
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "docker runner"
url = "http://gitlab.example.com"
token = "8wAr2hsHNsP4wBuSPEWD"
executor = "docker"
environment = ["GO111MODULE=on", "GOPROXY=https://goproxy.cn,direct"]
pre_clone_script = "eval \"$CI_PRE_CLONE_SCRIPT\""
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
pull_policy = ["always", "if-not-present"]
tls_verify = false
image = "busybox:1.28.4"
privileged = false
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
注意：实际环境中需要优化配置，修改配置后重启容器生效 docker restart gitlab-runner
访问页面 http://gitlab.example.com/admin/runners 查看注册的 runner
点击runner编辑勾选 Run untagged jobs ，让 runner 允许运行未使用 tag 标签选择的 job
3.4 测试 runner
首先创建组 g1 ，再创建项目 p1；
本地拉取项目，并添加 .gitlab-ci.yml 文件，内容如下：
stages:
- build
- deploy
variables:
projectname: "p1"
build:
stage: build
retry: 2
before_script:
- echo "before_script in build job"
script:
- df && ls -l
- echo ${projectname}
- echo "script in build job"
after_script:
- echo "after_script in build job"
tags:
- "docker"
deploy:
stage: deploy
script:
- echo "deploy"
when: manual
only:
- tags
提交代码
git add .gitlab-ci.yml
git commit -m "add .gitlab-ci.yml"
git push origin main
提交后 runner 会自动运行，但是此时会报错
Running with gitlab-runner 15.1.0 (98daeee0)
on docker runner 8wAr2hsH
Preparing the "docker" executor
00:03
Using Docker executor with image busybox:1.28.4 ...
Pulling docker image busybox:1.28.4 ...
Using docker image sha256:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a for busybox:1.28.4 with digest busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 ...
Preparing environment
00:00
Running on runner-8war2hsh-project-2-concurrent-0 via 935c6d59c5f6...
Getting source from Git repository
00:01
Fetching changes with git depth set to 20...
Reinitialized existing Git repository in /builds/g1/p1/.git/
fatal: unable to access 'http://gitlab.example.com/g1/p1.git/': Could not resolve host: gitlab.example.com
ERROR: Job failed: exit code 1
原因是 runner 创建的 docker 容器没有 gitlab.example.com 的 hosts 记录，无法拉起 git 仓库。
解决方法是设置 runner 的配置文件 config.toml 添加 extra_hosts 配置：
$ vi /srv/gitlab-runner/config/config.toml
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "docker runner"
url = "http://gitlab.example.com"
token = "8wAr2hsHNsP4wBuSPEWD"
executor = "docker"
environment = ["GO111MODULE=on", "GOPROXY=https://goproxy.cn,direct"]
pre_clone_script = "eval \"$CI_PRE_CLONE_SCRIPT\""
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
extra_hosts = ["gitlab.example.com:10.10.10.60"]
pull_policy = ["always", "if-not-present"]
tls_verify = false
image = "busybox:1.28.4"
privileged = false
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
# 重启容器生效
docker restart gitlab-runner
再次重试 job ，运行成功，流水线结果：
Running with gitlab-runner 15.1.0 (98daeee0)
on docker runner 8wAr2hsH
Preparing the "docker" executor
00:03
Using Docker executor with image busybox:1.28.4 ...
Pulling docker image busybox:1.28.4 ...
Using docker image sha256:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a for busybox:1.28.4 with digest busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 ...
Preparing environment
00:00
Running on runner-8war2hsh-project-2-concurrent-0 via 935c6d59c5f6...
Getting source from Git repository
00:01
Fetching changes with git depth set to 20...
Reinitialized existing Git repository in /builds/g1/p1/.git/
Checking out 43b70efe as main...
Skipping Git submodules setup
Executing "step_script" stage of the job script
00:00
Using docker image sha256:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a for busybox:1.28.4 with digest busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 ...
$ echo "before_script in build job"
before_script in build job
$ df && ls -l
Filesystem 1K-blocks Used Available Use% Mounted on
overlay 52403200 2873840 49529360 5% /
tmpfs 65536 0 65536 0% /dev
tmpfs 2001200 0 2001200 0% /sys/fs/cgroup
/dev/mapper/centos-root
52403200 2873840 49529360 5% /builds
/dev/mapper/centos-root
52403200 2873840 49529360 5% /cache
/dev/mapper/centos-root
52403200 2873840 49529360 5% /etc/resolv.conf
/dev/mapper/centos-root
52403200 2873840 49529360 5% /etc/hostname
/dev/mapper/centos-root
52403200 2873840 49529360 5% /etc/hosts
shm 65536 0 65536 0% /dev/shm
tmpfs 2001200 0 2001200 0% /proc/asound
tmpfs 2001200 0 2001200 0% /proc/acpi
tmpfs 65536 0 65536 0% /proc/kcore
tmpfs 65536 0 65536 0% /proc/keys
tmpfs 65536 0 65536 0% /proc/timer_list
tmpfs 65536 0 65536 0% /proc/sched_debug
tmpfs 2001200 0 2001200 0% /proc/scsi
tmpfs 2001200 0 2001200 0% /sys/firmware
total 4
-rw-rw-rw- 1 root root 6 Feb 22 01:41 README.md
$ echo ${projectname}
p1
$ echo "script in build job"
script in build job
Running after_script
00:00
Running after script...
$ echo "after_script in build job"
after_script in build job
Job succeeded
4. runner 使用 minio cache
runner 使用 cache 可以提高 ci 速度，这里使用兼容 s3 标准的 minio 为例。
4.1 安装 minio
minio 官方仓库：https://github.com/minio/minio
-
下载minio
wget https://dl.min.io/server/minio/release/linux-amd64/minio chmod +x minio mv minio /usr/local/bin/minio minio -v minio version RELEASE.2022-02-26T02-54-46Z -
下载minio client
wget https://dl.min.io/client/mc/release/linux-amd64/mc chmod +x mc mv mc /usr/local/bin/mc mc -v mc version RELEASE.2022-02-26T03-58-31Z -
配置minio数据目录
mkdir -p /opt/minio/data mkdir -p /opt/minio/logs -
创建 start_minio_server.sh 启动脚本
#!/bin/bash export MINIO_ROOT_USER=minio export MINIO_ROOT_PASSWORD=miniopwd nohup minio server /opt/minio/data > /opt/minio/logs/minio.log 2>&1 & -
启动minio server
chmod +x start_minio_server.sh ./start_minio_server.sh- 默认对外服务端口 9000
-
为客户端设置别名
mc alias set myminio http://10.10.10.60:9000 minio miniopwd -
配置存储桶
mc mb myminio/gitlab-cache
4.2 配置 runner
concurrent = 10
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "docker runner"
url = "https://gitlab.leffss.cn"
token = "xdfVPzdySQQNPCVgBxK1"
environment = ["GO111MODULE=on", "GOPROXY=https://goproxy.cn,direct"]
pre_clone_script = "eval \"$CI_PRE_CLONE_SCRIPT\""
executor = "docker"
[runners.custom_build_dir]
[runners.cache]
type = "s3"
shared = true
[runners.cache.s3]
ServerAddress = "10.10.10.60:9000"
AccessKey = "minio"
SecretKey = "miniopwd"
BucketName = "gitlab-cache"
# 使用 http 不安全方式连接
Insecure = true
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
#pull_policy = ["always", "if-not-present"]
pull_policy = ["if-not-present"]
extra_hosts = ["gitlab.leffss.cn:10.10.10.60"]
tls_verify = false
image = "busybox:1.28.4"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
重启 runner 生效
docker restart gitlab-runner
4.3 使用 cache
.gitlab-ci.yml 中使用 cache
stages:
- build
build:
image: maven:3.8.5-jdk-11
stage: build
cache:
paths:
- .m2/repository/
- target/
key: $CI_PROJECT_ID
script:
- mvn package
- ls -l target/*
# only:
# - merge_requests
