gitlab && gitlab-runner
192.168.77.101 gitlab rpm安装，url：http://192.168.77.101:8888
gitlab-ce安装：
打开：https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el7/ wget https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el7/gitlab-ce-16.8.5-ce.0.el7.x86_64.rpm yum install gitlab-ce-16.8.5-ce.0.el7.x86_64.rpm # 修改external_url # gitlab-ctl reconfigure
gitlab执行gitlab-ctl reconfigure 后卡在bruby_block[wait for logrotate service socket] action run解决方案
新打开一个窗口执行systemctl restart gitlab-runsvdir命令
由于之前安装gitlab对外开放端口：9090，后续和其他服务中途了端口，重启后一直失败，于是修改端口9090为8888，以下这些地方需要修改：
/etc/gitlab/gitlab.rb： external_url 'http://192.168.77.101:8888'
/var/opt/gitlab/gitlab-rails/etc/gitlab.yml：
gitlab:
##
gitlab-ctl restart、gitlab-ctl status、gitlab-ctl tail、gitlab-ctl tail redis、gitlab-ctl start redis、gitlab-ctl stop redis
gitlab-ctl reconfiguregitlab-ctl restart、gitlab-ctl status、gitlab-ctl tail、gitlab-ctl tail redis、gitlab-ctl start redis、gitlab-ctl stop redis
## Web server settings (note: host is the FQDN, do not include http://)
host: 192.168.77.101
port: 8888
gitlab_url: http://192.168.77.101:8888
external_k8s_proxy_url: http://192.168.77.101:8888/-/kubernetes-agent/k8s-proxy/
192.168.77.102 gitlab-runner docker安装：
[root@master ~]# mkdir -p /data/gitlab-runner/config # docker run -itd --restart always --name gitlab-runner -v /data/gitlab-runner/config:/etc/gitlab-runner -v /var/run/docker.sock:/var/run/docker.sock registry.cn-hangzhou.aliyuncs.com/jsonhc/gitlab-runner:v15.6.0
[root@node2 ~]# docker exec -it gitlab-runner /bin/bash
查看注册的runner：root@74ab08697613:/# gitlab-runner list
# gitlab-runner verify 容器内，检查注册的runner是否可以正常连接 # gitlab-runner list # gitlab-runner unregister --url http://gitlab.xxxx.com/ --token xxxxx # gitlab-runner unregister --name runner名称 使用名称注销，同名则删除第一个 # gitlab-runner unregister --all-runners 直接注销所有runner
runner注册：进入到项目，选择settings，选择CICD，选择runner，进行项目runner注册(还有admin设置那里进行注册)
在docker runner进行执行：
root@74ab08697613:/# gitlab-runner register --url http://192.168.77.101:8888 --token glrt-YJoao6aJ7y_wnK2gE8dy Runtime platform arch=amd64 os=linux pid=49 revision=44a1c2be version=15.6.0 WARNING: The 'register' command has been deprecated in GitLab Runner 15.6 and will be replaced with a 'deploy' command. For more information, see https://gitlab.com/gitlab-org/gitlab/-/issues/380872 Running in system-mode. Enter the GitLab instance URL (for example, https://gitlab.com/): [http://192.168.77.101:8888]: http://192.168.77.101:8888 Token specified trying to verify runner... WARNING: If you want to register use the '-r' instead of '-t'. Verifying runner... is alive runner=glrt-YJo Enter an executor: docker-ssh, ssh, docker-ssh+machine, instance, kubernetes, custom, docker, virtualbox, docker+machine, parallels, shell: docker Enter the default Docker image (for example, ruby:2.7): docker Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded! Configuration (with the authentication token) was saved in "/etc/gitlab-runner/config.toml"
最简单国内访问Docker Hub的替代方案
sudo tee /etc/docker/daemon.json <<EOF { "registry-mirrors": [ "https://docker.m.daocloud.io", "https://huecker.io", "https://dockerhub.timeweb.cloud", "https://noohub.ru" ] } EOF sudo systemctl daemon-reload sudo systemctl restart docker
测试：docker pull node:18-alpine3.14
gitlab cicd报错：参考：https://blog.csdn.net/mo_sss/article/details/136154855
error during connect: Post http://docker:2375/v1.40xxxxxx: dial tcp: lookup docker on 100.100.2.136:53: no such host
解决办法：
gitlab-runner：/etc/gitlab-runner/config.toml下面volumes新增：/var/run/docker.sock:/var/run/docker.sock，修改后如下：
volumes = ["/cache","/var/run/docker.sock:/var/run/docker.sock"]
以下成功案例：
stages: - build - deploy cache: paths: - ./dist build: image: node:16.13.0-alpine stage: build script: - npm config set registry https://registry.npmmirror.com - npm install - npm run build only: - main tags: - 192.168.77.102 artifacts: when: on_success paths: - ./dist deploy: image: docker:stable stage: deploy script: - ls -l && pwd - docker build -t test-web . only: - main tags: - 192.168.77.102
执行如下：
$ ls -l && pwd total 292 -rw-rw-rw- 1 root root 295 Jun 29 15:49 Dockerfile -rw-rw-rw- 1 root root 814 Jun 29 15:08 README.en.md -rw-rw-rw- 1 root root 6174 Jun 29 15:08 README.md drwxr-xr-x 4 root root 52 Jun 29 15:50 dist -rw-rw-rw- 1 root root 404 Jun 29 15:08 index.html -rw-rw-rw- 1 root root 97 Jun 29 15:08 jsconfig.json -rw-rw-rw- 1 root root 265909 Jun 29 15:51 package-lock.json -rw-rw-rw- 1 root root 1393 Jun 29 15:49 package.json drwxrwxrwx 3 root root 20 Jun 29 15:08 public drwxrwxrwx 8 root root 133 Jun 29 15:08 src -rw-rw-rw- 1 root root 936 Jun 29 15:08 vite.config.js
/builds/root/test-gitlab-ci
将上面的dist拷贝到gitlab-runner所在的宿主机，操作如下：
gitlab-runner：/etc/gitlab-runner/config.toml下面volumes新增：/data/gitlab-runner/data:/data，修改如下：
volumes = ["/cache","/var/run/docker.sock:/var/run/docker.sock","/data/gitlab-runner/data:/data"]
验证如下：
[root@node2 data]# pwd /data/gitlab-runner/data [root@node2 data]# ls -l dist/ total 12 drwxr-xr-x 2 root root 4096 Jun 29 12:48 assets drwxr-xr-x 2 root root 4096 Jun 29 12:48 images -rw-r--r-- 1 root root 492 Jun 29 12:48 index.html
或者将容器gitlab-runner的root下面的.ss进行挂载出来，然后copy到目标服务器，，这样也可以实现scp功能，参考：https://www.cnblogs.com/jimaojin/p/12611584.html
[root@node2 ~]# docker cp -a .ssh/ 74ab08697613:/root/ root@74ab08697613:/# ssh-copy-id -i /root/.ssh/id_rsa.pub root@192.168.77.101 # 输入101密码 root@74ab08697613:/# scp entrypoint root@192.168.77.101:/tmp/ # 此过程就不需要密码了 entrypoint
编辑.gitlab-ci.yml文件
stages: - test test: stage: test script: - echo "start to copy" - scp ./index.html root@192.168.77.101:/tmp only: - main tags: - 192.168.77.102
然后执行job：
然后去101上验证：
[root@node1 tmp]# ls -l total 4 -rw-r--r-- 1 root root 404 Jun 29 13:21 index.html
vm安装gitlab-runner：下载rpm：https://gitlab.com/gitlab-org/gitlab-runner/-/releases/v15.0.0
然后通过k8s调用，需要配置docker login信息，k8s config信息给用户runner：
[root@master ~]# wget https://gitlab.com/gitlab-org/gitlab-runner/-/releases/v15.0.0/downloads/packages/rpm/gitlab-runner_amd64.rpm [root@master ~]# yum install gitlab-runner_amd64.rpm [root@master ~]# systemctl status gitlab-runner [root@master ~]# gitlab-runner register --url http://192.168.77.101:8888 --token glrt-iq5YUr_hJ2xE4GxgbqcK [root@master ~]# gitlab-runner register --url http://192.168.77.101:8888 --token glrt-iq5YUr_hJ2xE4GxgbqcK Runtime platform arch=amd64 os=linux pid=12692 revision=febb2a09 version=15.0.0 Running in system-mode. Enter the GitLab instance URL (for example, https://gitlab.com/): [http://192.168.77.101:8888]: http://192.168.77.101:8888 Token specified trying to verify runner... WARNING: If you want to register use the '-r' instead of '-t'. Verifying runner... is alive runner=glrt-iq5 Enter an executor: docker, docker-ssh, parallels, ssh, docker+machine, docker-ssh+machine, custom, shell, virtualbox, kubernetes: shell Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded! [root@master ~]# usermod -aG docker gitlab-runner [root@master ~]# cp -a .docker /home/gitlab-runner/ [root@master ~]# cp -a .kube/ /home/gitlab-runner/ [root@master ~]# chown -R gitlab-runner:gitlab-runner /home/gitlab-runner/.docker [root@master ~]# chown -R gitlab-runner:gitlab-runner /home/gitlab-runner/.kube/
文件内容：
stages: - build - deploy build: stage: build script: - ls -l && pwd - docker build -t registry.cn-hangzhou.aliyuncs.com/jsonhc/springboot-manager . - docker push registry.cn-hangzhou.aliyuncs.com/jsonhc/springboot-manager only: - main tags: - k8s-192.168.77.103 deploy: stage: deploy script: - ls -l && pwd - kubectl set image deployment/springboot-manager springboot-manager=registry.cn-hangzhou.aliyuncs.com/jsonhc/springboot-manager -n testns only: - main tags: - k8s-192.168.77.103
[root@master ~]# yum install gitlab-runner_amd64.rpm [root@master ~]# systemctl status gitlab-runner [root@master ~]# gitlab-runner -v Version: 15.0.0 Git revision: febb2a09 Git branch: 15-0-stable GO version: go1.17.7 Built: 2022-05-19T19:34:08+0000 OS/Arch: linux/amd64
然后注册到gitlab：
[root@master ~]# gitlab-runner register --url http://192.168.77.101:8888 --token glrt-UH6AsSPmAHQKgExkRs4s Runtime platform arch=amd64 os=linux pid=14642 revision=febb2a09 version=15.0.0 Running in system-mode. Enter the GitLab instance URL (for example, https://gitlab.com/): [http://192.168.77.101:8888]: http://192.168.77.101:8888 Token specified trying to verify runner... WARNING: If you want to register use the '-r' instead of '-t'. Verifying runner... is alive runner=glrt-UH6 Enter an executor: ssh, virtualbox, docker+machine, docker-ssh+machine, kubernetes, shell, docker, docker-ssh, parallels, custom: shell Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!
然后gitlab查看注册的runner：
将100的公钥配置到gitlab：
.gitlab-ci.yml内容如下：
stages: - test test: stage: test script: - echo "start to copy to 192.168.77.100 " - cp ./index.html /tmp only: - main tags: - 192.168.77.100
自动执行后报错：fatal: git fetch-pack: expected shallow list
查询后是git版本太低导致：
[root@master ~]# git --version git version 1.8.3.1
解决办法升级一下git版本：
[root@master ~]# yum install http://opensource.wandisco.com/centos/7/git/x86_64/wandisco-git-release-7-1.noarch.rpm [root@master ~]# yum install git [root@master ~]# git --version git version 2.41.0
再次执行后：
100这台服务器验证：
[root@master ~]# ls -l /tmp/ total 8 -rw-rw-r-- 1 gitlab-runner gitlab-runner 404 Jun 29 14:18 index.html
gitlab-runner executor为shell时，当前工作目录就是gitlab-runner这台vm家目录下面：
.gitlab-ci.yml
stages: - build build: image: node:16.13.0-alpine stage: build script: - pwd && ls -l only: - main tags: - 192.168.77.100 artifacts: when: on_success paths: - ./dist
执行如下：
然后看100这一台服务器：
两者对应一致，如果想在shell这个executor下进行npm编译，则需要在100这台服务器下进行安装相应的环境：node、npm等等
而上面的：image: node:16.13.0-alpine则在shell环境下没有作用
报错信息：参考：https://www.cnblogs.com/netonline/p/9800159.html
报错信息：fatal: unable to access 'https://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx@gitlab.x.com/root/cmop.git/': Peer's Certificate issuer is not recognized.
# 因runner运行时的执行者是gitlab-runner账户，需要在gitlab-runner账号下设置访问https类网站时，免验证 [root@gitlab-runner ~]# su - gitlab-runner [gitlab-runner@gitlab-runner ~]$ git config --global http."sslVerify" false # 查看 [gitlab-runner@gitlab-runner ~]$ cat /home/gitlab-runner/.gitconfig [http] sslVerify = false
因为gitlab使用了不信任的证书，所以gitlab-runner用户clone的时候，进行关闭ssl校验
报错信息：Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.27/info: dial unix /var/run/docker.sock: connect: permission denied
原因：gitlab-runner账号权限不足，不能访问/var/run/docker.sock
-bash-4.2$ docker ps permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.45/containers/json": dial unix /var/run/docker.sock: connect: permission denied -bash-4.2$ docker ps permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.45/containers/json": dial unix /var/run/docker.sock: connect: permission denied
解决办法：
[root@node2 yum.repos.d]# usermod -aG docker gitlab-runner [root@node2 yum.repos.d]# groups gitlab-runner gitlab-runner : gitlab-runner docker
-bash-4.2$ exit logout [root@node2 ~]# su - gitlab-runner Last login: Mon Jul 29 05:30:32 EDT 2024 on pts/1 -bash-4.2$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
gitlab-runner注册的时候报错：
报错信息：ERROR: Registering runner... failed runner=xxxxxxx status=couldn't execute POST against https://x.x.x.x/api/v4/runners: Post https://x.x.x.x/api/v4/runners: x509: cannot validate certificate for x.x.x.x because it doesn't contain any IP SANs
因为gitlab使用了不信任的证书，所以gitlab-runner注册的时候，需要进行证书指定：
解决办法：将证书从浏览器下载下来，注册的时候指定，使用"--tls-ca-file"参数，指定自签名的ca根证书
.gitlab-ci.yaml：
before_script:
- apt-get install rubygems ruby-dev -y
## before_script属性将在运行任何内容之前为你的应用安装依赖，一个名为run-test的job，将打印当前系统的ruby版本，二者共同构成了在每次推送到仓库的任何分支时都会被触发的pipeline
- stages:pipeline的阶段列表，定义整个pipeline阶段
- stage:定义某个job的所在阶段
- image:指定一个基础Docker进行作为基础运行环境，比如:node,python,java
- tags:用于指定Runner，tags的取值范围是在该项目可惜可见的runner tags中，也就是前面我们设置的那个tag
- only/except:知道当前任务条件
- when:实现在发生故障时仍能运行的作业
- cache:讲当前工作环境目录中的一些文件，文件夹存储起来，用于在各个任务初始化的时候恢复
- environment:指定部署相关任务的环境，并非真实环境，是对要部署到某环境的任务的归类。方便在gitlab上聚合以便进行回滚和重新部署操作
- artifacts:保留文档。在每次 job 之前runner会清除未被 git 跟踪的文件。为了让编译或其他操作后的产物可以留存到后续使用，添加该参数并设置保留的目录，保留时间等。被保留的文件将被上传到gitlab以备后续使用。
- dependencies:任务依赖。指定job的前置job。添加该参数后，可以获取到前置job的artifacts。注意如果前置 job 执行失败，导致没能生成artifacts，则 job 也会直接失败。
image: "ruby:2.5" before_script: - apt-get update -qq && apt-get install -y -qq sqlite3 libsqlite3-dev nodejs - ruby -v - which ruby - gem install bundler --no-document - bundle install --jobs ${nproc} "${FLAGS[@]}" rspec: script: - bundle exec rspec rubocop: script: - bundle exec rubocop
yml样例：
variables: SERVER_PORT: 8018 PACKAGE_COMMAND: yarn install RUN_COMMAND: sudo yarn build:stage PRO_RUN_COMMAND: sudo yarn build:prod NGINX_PATH=/usr/share/nginx/html/8108 PROJECT_NAME: omsSystem stages: - build - probuild - deploy - uploadfile build: stage: build script: - echo "Building the app" - $PACKAGE_COMMAND - $RUN_COMMAND artifacts: expire_in: 10 day paths: - ./dist only: - test tags: - ci-test probuild: stage: probuild scrpt: - echo "Building the app" - $PACKAGE_COMMAND - $PRO_RUN_COMMAND artifacts: expire_in: 10 day paths: - ./dist only: - master tags: - ci-test uploadfile: stage: uploadfile script: - pwd - cd dist - zip -q -r ${PROJECT_NAME}front.zip - mv ${PROJECT_NAME}front.zip /usr/share/nginx/download only: - master tags: - ci-test
或者另一个样例参考：
variables: SERVER_PORT: 8055 JAR_PATH: target/omsSystem.jar PROJECT_NAME: omsSystem WORK_PATH: /home/java stages: - build - test - deploy - uploadfile build: stage: build script: - echo "Building the app" - echo "项目开始打包----------------------" - mvn clean package -Dmaven.test.skip=true artifacts: expire_in: 10 day paths: - ${JAR_PATH} only: - test - master tags: - ci-test test: stage: test script: - mvn test only: - test tags: - ci-test deploy: stage: deploy script: - cp ${JAR_PATH} ${WORK_PATH} - ./start_java.sh start ${PROJECT_NAME}.jar only: - test tags: - ci-test uploadfile: stage: uploadfile script: - pwd - cp ${JAR_PATH} /usr/share/nginx/download only: - master tags: - ci-test
当docker启动gitlab时，需要将gitlab的ssh 22端口配置一下并暴露出来：
配置ssh访问
进入容器中之后，我们点击头像，选择Preferences,可以看到左侧有ssh keys选项，导入自己的公钥，然后就可以通过ssh的方式，来拉取项目了。
$ git clone git@192.168.0.254:gwk/flow.git Cloning into 'flow'... git@192.168.0.254: Permission denied (publickey,gssapi-keyex,gssapi-with-mic). fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists.
出现这个问题，主要是因为我们的gitlab是运行在docker中的，而我们项目生成的git地址是宿主机的地址，这样肯定是无法访问的。
我们在构建容器的时候，将容器中的ssh端口，映射到了宿主机的30022端口上，因此我们需要做如下操作：
# 进入容器，修改 /etc/gitlab/gitlab.rb root@192:/# cat /etc/gitlab/gitlab.rb|grep gitlab_shell_ssh_port gitlab_rails['gitlab_shell_ssh_port'] = '30022' 将 "gitlab_shell_ssh_port" 更改为我们的宿主机映射地址，然后执行 root@192:/# gitlab-ctl reconfigure
刷新页面，可看到在ssh地址中多了一个端口，克隆地址就变成如下：
ssh://git@192.168.0.254:30022/gwk/flow.git
启动命令：
docker run --detach \
--hostname 192.168.1.205 \
--publish 9443:443 --publish 9080:9080 --publish 30022:22 \
--name gitlab \
--restart always \
--volume $GITLAB_HOME/config:/etc/gitlab:Z \
--volume $GITLAB_HOME/logs:/var/log/gitlab:Z \
--volume $GITLAB_HOME/data:/var/opt/gitlab:Z \
--shm-size 256m \
gitlab/gitlab-ee:latest
或者
docker run -d -p 2443:443 -p 5678:80 -p 2222:22 --name gitlab --restart always -v /srv/gitlab/config:/etc/gitlab -v /srv/gitlab/logs:/var/log/gitlab -v /src/gitlab/data:/var/opt/gitlab docker.io/gitlab/gitlab-ce
修改gitlab.rb文件
vim /srv/gitlab/config/gitlab.rb
external_url 'http://172.20.9.3:5678'
gitlab_rails['gitlab_ssh_host'] = '172.20.9.3'
gitlab_rails['gitlab_shell_ssh_port'] = '2222'
# 停止并移除之前启动的GitLab容器
docker rm -f gitlab
# 重启启动
docker run -d -p 2443:443 -p 5678:5678 -p 2222:22 --name gitlab --restart always -v /srv/gitlab/config:/etc/gitlab -v /srv/gitlab/logs:/var/log/gitlab -v /src/gitlab/data:/var/opt/gitlab docker.io/gitlab/gitlab-ce
docker部署：
- if [ $(docker ps -aq --filter name=app-container) ]; then docker rm -f app-container;fi - docker run -d -p 8082:80 --name app-container appimages
当部署gitlab出现其中的puma服务出现端口8080占用时，需要修改puma端口：
[root@node1 ~]# netstat -antp|grep -i listen|grep 8085 [root@node1 ~]# vim /etc/gitlab/gitlab.rb 修改这个配置文件8080为8085 [root@node1 ~]# grep "8085" /etc/gitlab/gitlab.rb puma['port'] = 8085 [root@node1 ~]# vim /var/opt/gitlab/gitlab-rails/etc/puma.rb 修改这个配置文件8080为8085 [root@node1 ~]# grep "8085" /var/opt/gitlab/gitlab-rails/etc/puma.rb bind 'tcp://127.0.0.1:8085' [root@node1 ~]# gitlab-ctl reconfigure 重新生成配置
然后重启服务并查看监听端口：
[root@node1 ~]# gitlab-ctl restart [root@node1 ~]# netstat -antp|grep -i listen|grep 8085 tcp 0 0 127.0.0.1:8085 0.0.0.0:* LISTEN 50668/puma 6.4.0 (u
