多种方式为极狐 gitlab 安装 runner
多种方式为极狐 gitlab 安装 runner
1. 环境信息
1.1 主机信息
| 主机 | 用途 |
|---|---|
| 10.10.10.60 | 极狐gitlab v14.10.0 |
- 域名 gitlab.example.com，指向 10.10.10.60，这里使用设置本机 hosts 的方式解析，如果有条件的话使用本地 dns 服务器设置解析
- 开启了 san 证书的 https，ca.pem 为 ca 根证书
2. linux 上安装
参考官方文档：Install GitLab Runner | GitLab
系统：centos 7.9 x64
2.1 二进制包方式
runner 发布地址：https://gitlab.com/gitlab-org/gitlab-runner/-/tags
根据需求下载对应版本，这里下载 v14.8.0 版本：
wget https://gitlab.com/gitlab-org/gitlab-runner/-/releases/v14.8.0/downloads/binaries/gitlab-runner-linux-amd64
mv gitlab-runner-linux-amd64 /usr/local/bin/gitlab-runner
chmod +x /usr/local/bin/gitlab-runner
添加用户
useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash
安装启动服务
gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner
gitlab-runner start
注册 runner，参考官方文档：Registering runners | GitLab
$ gitlab-runner register --tls-ca-file=/etc/gitlab-runner/ca.pem
Runtime platform arch=amd64 os=linux pid=1594 revision=565b6c0b version=14.8.0
Running in system-mode.
Enter the GitLab instance URL (for example, https://gitlab.com/):
https://gitlab.example.com
Enter the registration token:
_xYaYy6WPUqsMm2wVLsV
Enter a description for the runner:
[localhost.localdomain]: docker runner
Enter tags for the runner (comma-separated):
docker
Enter optional maintenance note for the runner:
docker runner
Registering runner... succeeded runner=_xYaYy6W
Enter an executor: parallels, shell, docker+machine, virtualbox, docker-ssh+machine, kubernetes, custom, docker, docker-ssh, ssh:
shell
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!
# 重启 gitlab runner
systemctl restart gitlab-runner
2.2 rpm 或者 deb 包方式
方法和二进制方式类似，只需下载安装包时下载对应的 rpm 和 deb 即可，这里就不演示了。
更新方式：
# 下载新版本的 gitlab runner rpm 包
rpm -Uvh gitlab-runner_<arch>.rpm
随后重启 gitlab-runner 即可
3. docker 方式
3.1 创建配置目录
mkdir -p /srv/gitlab-runner/config
cp ca.pem /srv/gitlab-runner/config
3.2 启动 runner
docker run -d --name gitlab-runner --restart always \
--env TZ=Asia/Shanghai \
--add-host=gitlab.example.com:10.10.10.60 \
-v /srv/gitlab-runner/config:/etc/gitlab-runner \
-v /var/run/docker.sock:/var/run/docker.sock \
gitlab/gitlab-runner:v14.7.0
- mac 本机配置了 hosts，那么 hosts 配置对 docker 容器里面的所有容器都适用，但是在 linux 的时候就不适用了，所以这里使用 --add-host 添加容器里的 hosts 解析
3.3 注册 runner
docker run --rm -it --add-host=gitlab.example.com:10.10.10.60 -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner:v14.7.0 register --tls-ca-file=/etc/gitlab-runner/ca.pem
docker restart gitlab-runner
- 根据提示填写 gitlab 地址，以及 token 等信息
查看配置文件
$ cat /srv/gitlab-runner/config/config.toml
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "docker runner"
url = "https://gitlab.example.com"
token = "CGcmbaHn1fHGFX7ViHuF"
tls-ca-file = "/etc/gitlab-runner/ca.pem"
executor = "docker"
environment = ["GO111MODULE=on", "GOPROXY=https://goproxy.cn,direct"]
pre_clone_script = "eval \"$CI_PRE_CLONE_SCRIPT\""
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
pull_policy = ["always", "if-not-present"]
extra_hosts = ["gitlab.example.com:10.10.10.60"]
tls_verify = false
image = "busybox:1.28.4"
privileged = false
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
注意：实际环境中需要优化配置，修改配置后重启容器生效 docker restart gitlab-runner
更新方式：
docker pull gitlab/gitlab-runner:latest
docker stop gitlab-runner && docker rm gitlab-runner
docker run -d --name gitlab-runner --restart always \
--env TZ=Asia/Shanghai \
--add-host=gitlab.example.com:10.10.10.60 \
-v /srv/gitlab-runner/config:/etc/gitlab-runner \
-v /var/run/docker.sock:/var/run/docker.sock \
gitlab/gitlab-runner:latest
4. windows 上安装
系统：windows 10 64位
4.1 下载程序
runner 发布地址：https://gitlab.com/gitlab-org/gitlab-runner/-/tags
4.2 注册 runner
解压程序包到 C:\，使用 PowerShell 执行程序（请勿使用 PowerShell ISE 执行，会报错）
4.3 安装 runner 为 windows 服务
首先以管理员方式允许 PowerShell
4.4 测试 windows runner
添加 .gitlab-ci.yml
stages:
- build
variables:
projectname: "p1"
build:
stage: build
retry: 2
script:
- pwd
- get-host
tags:
- "windows"
运行 runner 失败
Running with gitlab-runner 14.7.0 (98daeee0)
on windows runner xhyz8oiW
Preparing the "shell" executor
00:00
Using Shell executor...
Preparing environment
00:00
ERROR: Job failed (system failure): prepare environment: failed to start process: exec: "pwsh": executable file not found in %PATH%. Check https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading for more information
解决方法：gitlab-runner: prepare environment failed to start process pwsh in windows - Stack Overflow
再次运行成功：
Running with gitlab-runner 14.7.0 (98daeee0)
on windows runner xhyz8oiW
Preparing the "shell" executor
00:00
Using Shell executor...
Preparing environment
00:01
Running on P_FAFULI-NB1...
Getting source from Git repository
Fetching changes with git depth set to 20...
Initialized empty Git repository in C:/gitlab-runner-windows-amd64/builds/xhyz8oiW/0/g1/p1/.git/
Created fresh repository.
Checking out 88ac867c as main...
git-lfs/2.13.3 (GitHub; windows amd64; go 1.16.2; git a5e65851)
Skipping Git submodules setup
Executing "step_script" stage of the job script
00:01
$ pwd
Path
----
C:\gitlab-runner-windows-amd64\builds\xhyz8oiW\0\g1\p1
$ get-host
Name : ConsoleHost
Version : 5.1.19041.1320
InstanceId : 289c4b2c-386e-452a-80d3-dbd947609adc
UI : System.Management.Automation.Internal.Host.InternalHostUserInterface
CurrentCulture : zh-CN
CurrentUICulture : zh-CN
PrivateData : Microsoft.PowerShell.ConsoleHost+ConsoleColorProxy
DebuggerEnabled : True
IsRunspacePushed : False
Runspace : System.Management.Automation.Runspaces.LocalRunspace
Cleaning up project directory and file based variables
00:00
Job succeeded
4.5 更新 runner
cd C:\gitlab-runner-windows-amd64
.\gitlab-runner-windows-amd64.exe stop
下载新版本覆盖，然后重新启动
cd C:\gitlab-runner-windows-amd64
.\gitlab-runner-windows-amd64.exe start
4.6 卸载 runner
cd C:\gitlab-runner-windows-amd64
.\gitlab-runner-windows-amd64.exe stop
.\gitlab-runner-windows-amd64.exe uninstall
# 然后删除目录 gitlab-runner-windows-amd64 即可
5. Kubernetes 上安装
官方文档：GitLab Runner Helm Chart | GitLab
5.1 安装 docker
参考：Get Docker | Docker Documentation 或者
echo "
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1" >> /etc/sysctl.conf
sysctl -p
curl https://releases.rancher.com/install-docker/19.03.sh | sh
systemctl enable docker
设置国内源
vi /etc/docker/daemon.json
{
"registry-mirrors": ["https://hccwwfjl.mirror.aliyuncs.com"]
}
systemctl restart docker
- 其他源：http://hub-mirror.c.163.com，建议用这个 163 的
5.2 安装 k3s
参考：k8s 测试环境搭建（k3s） - leffss - 博客园 (cnblogs.com)
curl -sfL http://rancher-mirror.cnrancher.com/k3s/k3s-install.sh | INSTALL_K3S_VERSION=v1.20.7+k3s1 INSTALL_K3S_MIRROR=cn sh -s - --docker
$ systemctl status k3s
● k3s.service - Lightweight Kubernetes
Loaded: loaded (/etc/systemd/system/k3s.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2022-02-22 13:38:40 CST; 59min ago
Docs: https://k3s.io
Process: 2247 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
Process: 2245 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
Main PID: 2250 (k3s-server)
Tasks: 18
Memory: 638.6M
CGroup: /system.slice/k3s.service
└─2250 /usr/local/bin/k3s server
$ kubectl get node
NAME STATUS ROLES AGE VERSION
localhost.localdomain Ready control-plane,master 60m v1.20.7+k3s1
$ kubectl get pod -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system helm-install-traefik-g4xsp 0/1 Completed 0 59m
kube-system metrics-server-86cbb8457f-8wghh 1/1 Running 0 59m
kube-system local-path-provisioner-5ff76fc89d-r87zg 1/1 Running 0 59m
kube-system svclb-traefik-24nj7 2/2 Running 0 59m
kube-system coredns-854c77959c-nm28h 1/1 Running 0 59m
kube-system traefik-6f9cbd9bd4-lm4xf 1/1 Running 0 59m
5.3 安装 helm v3
安装 helm v3（v3.3.1 或者更新），参考官方安装文档：Helm | Installing Helm
这里使用手动下载安装方式：
# 发布地址：https://github.com/helm/helm/releases
$ curl -O https://get.helm.sh/helm-v3.8.0-linux-amd64.tar.gz
$ tar zxvf helm-v3.8.0-linux-amd64.tar.gz
$ mv linux-amd64/helm /usr/local/bin/helm
$ chmod +x /usr/local/bin/helm
$ helm version
version.BuildInfo{Version:"v3.8.0", GitCommit:"d14138609b01886f544b2025f5000351c9eb092e", GitTreeState:"clean", GoVersion:"go1.17.5"}
5.4 添加 gitlab repo
helm repo add gitlab https://charts.gitlab.io/
# 查看完整的镜像仓库列表
helm search repo -l gitlab/gitlab-runner
5.5 安装 runner chart
首先准备配置文件 values.yaml ，官方模板文件：values.yaml · main · GitLab.org / charts / GitLab Runner · GitLab
vi values.yaml
imagePullPolicy: IfNotPresent
terminationGracePeriodSeconds: 3600
concurrent: 10
checkInterval: 30
certsSecretName: gitlab.example.com.crt
# gitlab 地址
gitlabUrl: https://gitlab.example.com
# runner 注册 token
runnerRegistrationToken: "_xYaYy6WPUqsMm2wVLsV"
runners:
# runner 的配置文件
config: |
[[runners]]
[runners.kubernetes]
namespace = "{{.Release.Namespace}}"
image = "busybox:1.28.4"
rbac:
create: true
其中 certsSecretName 保存 gitlab 的 https 证书，名称格式为：
kubectl create secret generic gitlab.example.com.crt --from-file=gitlab.example.com.crt
开始创建
# 格式：helm install --namespace <NAMESPACE> gitlab-runner -f <CONFIG_VALUES_FILE> gitlab/gitlab-runner
helm install --version 0.37.2 gitlab-runner -f values.yaml gitlab/gitlab-runner
- --version 指定 chart 版本，0.37.2 对应 v14.7.0 的 runner
第一次运行会报错
Error: INSTALLATION FAILED: Kubernetes cluster unreachable: Get "http://localhost:8080/version": dial tcp [::1]:8080: connect: connection refused
原因是：helm v3 版本不再需要 Tiller，而是直接访问 ApiServer 来与 k8s 交互，通过环境变量 KUBECONFIG 来读取存有 ApiServer 的地址与 token 的配置文件地址，默认地址为 ~/.kube/config，但是 k3s 的 KUBECONFIG 默认为 /etc/rancher/k3s/k3s.yaml
解决方法：手动配置 KUBECONFIG 环境变量
1）临时解决：
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
2）永久解决：
$ vi /etc/profile # 添加
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
$ source /etc/profile
再次运行
$ helm install --version 0.37.2 gitlab-runner -f values.yaml gitlab/gitlab-runner
NAME: gitlab-runner
LAST DEPLOYED: Thu Feb 24 16:52:34 2022
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Your GitLab Runner should now be registered against the GitLab instance reachable at: "https://gitlab.example.com"
Runner namespace "default" was found in runners.config template.
查看 runner 的 pod 日志，会发现报错：
$ kubectl logs gitlab-runner-gitlab-runner-76fdff5f4f-mf66w
...
ERROR: Registering runner... failed runner=_xYaYy6W status=couldn't execute POST against https://gitlab.example.com/api/v4/runners: Post "https://gitlab.example.com/api/v4/runners": dial tcp: lookup gitlab.example.com on 10.43.0.10:53: no such host
PANIC: Failed to register the runner. You may be having network problems.
原因是 runner 的 pod 无法解析 gitlab.example.com，解决方法如下：
在集群 coredns 中添加 hosts 解析
$ kubectl -n kube-system edit configmaps coredns
...
apiVersion: v1
data:
Corefile: |
.:53 {
errors
health
ready
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure
fallthrough in-addr.arpa ip6.arpa
}
hosts /etc/coredns/NodeHosts {
10.10.10.60 gitlab.example.com
ttl 60
reload 15s
fallthrough
}
...
# 删除 coredns pod 生效
$ kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
metrics-server-86cbb8457f-hqt9p 1/1 Running 0 5h6m
local-path-provisioner-5ff76fc89d-cz9hj 1/1 Running 0 5h6m
helm-install-traefik-lmt7r 0/1 Completed 0 5h6m
coredns-854c77959c-w4t7j 1/1 Running 0 5h6m
svclb-traefik-fp7f9 2/2 Running 0 5h5m
traefik-6f9cbd9bd4-l7h45 1/1 Running 0 5h5m
$ kubectl -n kube-system delete pod coredns-854c77959c-w4t7j
pod "coredns-854c77959c-nm28h" deleted
再次查看，runner 已经正常运行
$ kubectl get pod
NAME READY STATUS RESTARTS AGE
gitlab-runner-gitlab-runner-76fdff5f4f-mf66w 1/1 Running 1 4m50s
注册成功
测试任务也成功
Running with gitlab-runner 14.7.0 (98daeee0)
on gitlab-runner-gitlab-runner-76fdff5f4f-mf66w c3qaY9MY
Preparing the "kubernetes" executor
00:00
Using Kubernetes namespace: default
Using Kubernetes executor with image busybox:1.28.4 ...
Using attach strategy to execute scripts...
Preparing environment
00:18
Waiting for pod default/runner-c3qay9my-project-2-concurrent-0sdnb8 to be running, status is Pending
Waiting for pod default/runner-c3qay9my-project-2-concurrent-0sdnb8 to be running, status is Pending
ContainersNotInitialized: "containers with incomplete status: [init-permissions]"
ContainersNotReady: "containers with unready status: [build helper]"
ContainersNotReady: "containers with unready status: [build helper]"
Waiting for pod default/runner-c3qay9my-project-2-concurrent-0sdnb8 to be running, status is Pending
ContainersNotInitialized: "containers with incomplete status: [init-permissions]"
ContainersNotReady: "containers with unready status: [build helper]"
ContainersNotReady: "containers with unready status: [build helper]"
Waiting for pod default/runner-c3qay9my-project-2-concurrent-0sdnb8 to be running, status is Pending
ContainersNotInitialized: "containers with incomplete status: [init-permissions]"
ContainersNotReady: "containers with unready status: [build helper]"
ContainersNotReady: "containers with unready status: [build helper]"
Waiting for pod default/runner-c3qay9my-project-2-concurrent-0sdnb8 to be running, status is Pending
ContainersNotReady: "containers with unready status: [build helper]"
ContainersNotReady: "containers with unready status: [build helper]"
Waiting for pod default/runner-c3qay9my-project-2-concurrent-0sdnb8 to be running, status is Pending
ContainersNotReady: "containers with unready status: [build helper]"
ContainersNotReady: "containers with unready status: [build helper]"
Running on runner-c3qay9my-project-2-concurrent-0sdnb8 via gitlab-runner-gitlab-runner-76fdff5f4f-mf66w...
Getting source from Git repository
00:01
Fetching changes with git depth set to 20...
Initialized empty Git repository in /builds/g1/p1/.git/
Created fresh repository.
Checking out 1c6e29cb as main...
Skipping Git submodules setup
Executing "step_script" stage of the job script
00:00
$ df
Filesystem 1K-blocks Used Available Use% Mounted on
overlay 52403200 3039852 49363348 6% /
tmpfs 65536 0 65536 0% /dev
tmpfs 8185140 0 8185140 0% /sys/fs/cgroup
/dev/mapper/centos-root
52403200 3039852 49363348 6% /scripts-2-6
/dev/mapper/centos-root
52403200 3039852 49363348 6% /logs-2-6
/dev/mapper/centos-root
52403200 3039852 49363348 6% /builds
/dev/mapper/centos-root
52403200 3039852 49363348 6% /dev/termination-log
/dev/mapper/centos-root
52403200 3039852 49363348 6% /etc/resolv.conf
/dev/mapper/centos-root
52403200 3039852 49363348 6% /etc/hostname
/dev/mapper/centos-root
52403200 3039852 49363348 6% /etc/hosts
shm 65536 0 65536 0% /dev/shm
tmpfs 8185140 12 8185128 0% /var/run/secrets/kubernetes.io/serviceaccount
tmpfs 8185140 0 8185140 0% /proc/acpi
tmpfs 65536 0 65536 0% /proc/kcore
tmpfs 65536 0 65536 0% /proc/keys
tmpfs 65536 0 65536 0% /proc/timer_list
tmpfs 65536 0 65536 0% /proc/sched_debug
tmpfs 8185140 0 8185140 0% /proc/scsi
tmpfs 8185140 0 8185140 0% /sys/firmware
$ echo ${projectname}
p1
$ date
Thu Feb 24 09:00:31 UTC 2022
Cleaning up project directory and file based variables
00:01
Job succeeded
