Docker安装GitLab与Runner(网关),常规设置,自动化用到k8s+token
[转]图文详解k8s自动化持续集成之GitLab CI/CD
Windows里面使用Debian命令行工具完成
和Docker网络相关的命令
查看某一个容器的网络 docker inspect 容器ID 查看docker当前网络 docker network ls 首先创建一个网络(下面通过docker-compose的yml文件创建,这里只做命令的了解) docker network create gitlab-network 将现有的容器连到创建的网络中(每个容器都要连到这个网络里) docker network connect gitlab-network gitlab-server #容器名 docker network connect gitlab-network gitlab-runner 查看网络内的容器信息 docker network inspect gitlab-network 也可以在运行时指定镜像 docker run --network gitlab-network 镜像名称 移除指定的网络 docker network rm gitlab-network
先删掉下面要用的bridge名称
使用Docker Compose安装GitLab-ce和GitLab-runner
Docker Compose で GitLab + GitLab Runner の環境を整える(Window10) 重要参考
Win10下使用Docker搭建Gitlab CI自动构建平台
docker rm & docker rmi & docker prune 的差异
- docker rm : 删除一个或多个 容器
- docker rmi : 删除一个或多个 镜像
- docker prune: 用来删除不再使用的 docker 对象
docker+gitlab+gitlab-runner部署(CentOs7)重要参考
Docker Compose yml for Gitlab and Gitlab Runner
SMTP settings(备用参考)
#vim /etc/gitlab/gitlab.rb
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtp.exmail.qq.com"
gitlab_rails['smtp_port'] = 465
gitlab_rails['smtp_user_name'] = "developers@aaa.net"
gitlab_rails['smtp_password'] = "Del43@\#$@1"
gitlab_rails['smtp_domain'] = "exmail.qq.com"
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_enable_starttls_auto'] = true
gitlab_rails['smtp_tls'] = true
gitlab_rails['gitlab_email_from'] = "developers@aaa.net"
测试邮件
#gitlab-ctl reconfigure
#gitlab-rails console
#Notify.test_email('315360007@qq.com', 'Message Subject', 'Message Body').deliver_now
配置本地windows的host文件:
127.0.0.1 gitlab-server
127.0.0.1 nexus3
浏览器访问:http://gitlab-server:9080/ 或者 http://ip:9080/
用户名:root,密码:自己设置
用root用户登录
创建测试项目:mvcdockerrunner1
添加SSH密钥
本地windows需要在当前登录用户的目录里面找到.ssh目录下面的****.pub文件添加公钥,同时设置config文件
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
Host gitlab-server HostName gitlab-server User git PreferredAuthentications publickey IdentityFile ~/.ssh/315360007 Port 23
克隆代码
手动设置激活的Runner(很重要)
1、注册gitlab-runner(按照提示一步一步做),熟悉了直接用下面第二种
docker exec -it gitlab-runner gitlab-runner register
2. 我们会输入 http://gitlab.local.net:9080 或 http://ip:port 也就是我们安装在本地的GitLab
3. Please enter the gitlab-ci token for this runner 要求输入 gitlab-ci token
在项目的 管理区域->runners中可以找到(这里注册的是share类型runner)
4. 输入描述,如:gitlab-runner
5. 输入tag(留空也可以,之后可以进行编辑 )
6. 选择当遇到没有打标签的提交时是否会执行,我们选 true
7. 是否锁定此runner 到当前项目, 我们选 false
8. 选一个执行者 executor
这一步比较重要 (ssh, docker+machine, docker-ssh+machine, kubernetes, docker, parallels, virtualbox, docker-ssh, shell)
我们选docker
9. 选择默认使用的镜像: docker:stable
2、注册gitlab-runner(直接传参数一次完成)
# 手动设置激活的Runner docker exec -it gitlab-runner gitlab-runner register \ --non-interactive \ --executor "docker" \ --docker-image docker:stable \ --url "http://gitlab-server:9080/" \ --registration-token "K6BiurzDBuzx23a-mV-f" \ --description "gitlab-runner" \ --tag-list "docker,company" \ --run-untagged="true" \ --locked="false" \ --docker-privileged="true" \ --docker-extra-hosts "gitlab-server:172.20.0.1" \ --docker-extra-hosts "nexus3:172.20.0.1" #解释很重要 #"docker:stable"这里之所以这样是因为基于这个镜像它包含了docker等工具,可以在gitlab-runner执行的.gitlab-ci.yml中有docker指令时 #而无需再安装docker 参考地址:https://docs.gitlab.com/ee/ci/docker/using_docker_build.html #privileged=true #使用docker-in-docker时通常为true #extra-hosts #设置gitlab-server的网关地址
进入 gitlab-runner的容器里
docker exec -it runner-container-id /bin/bash
查看/etc/gitlab-runner/config.toml文件
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
cat /etc/gitlab-runner/config.toml concurrent = 1 check_interval = 0 [session_server] session_timeout = 1800 [[runners]] name = "gitlab-runner" url = "http://gitlab-server:9080/" token = "mWpRt6ry7HimmAyt86T1" executor = "docker" [runners.custom_build_dir] [runners.cache] [runners.cache.s3] [runners.cache.gcs] [runners.cache.azure] [runners.docker] tls_verify = false image = "docker:stable" privileged = true disable_entrypoint_overwrite = false oom_kill_disable = false disable_cache = false volumes = ["/cache"] extra_hosts = ["gitlab-server:172.20.0.1"] shm_size = 0
五、提交项目代码完成CI/CD
Gitlab CI/CD 打包过程报 “ERROR: Uploading artifacts to coordinator... too large” 如何解决
Uploading artifacts for successful job 00:13 Uploading artifacts... mevdockerrunner1.tar: found 1 matching files and directories ERROR: Uploading artifacts as "archive" to coordinator... too large archive id=34 responseStatus=413 Request Entity Too Large status=413 token=fhcnwRyE FATAL: too large
GitLab管理中心里设置CI/CD持续集成和部署里最大产物大小
设置和使用Gitlab-ci变量连接Kubernetes:
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
1 .kube_deploy: 2 image: boxboat/kubectl:1.17.4 3 variables: 4 HELM_TOKEN: ${HELM_USER_TOKEN} 5 K8S_API_SERVER: ${KUBE_API_ADDRESS} 6 K8S_API_SERVER_CERT: ${KUBE_API_CA} 7 before_script: 8 - mkdir -p /home/alpine/.kube 9 - cp .ci/kube-config-template /home/alpine/.kube/config 10 - sed -i "s/{{ HELM_USER_TOKEN }}/${HELM_TOKEN}/g" /home/alpine/.kube/config 11 - sed -i "s/{{ KUBE_API_ADDRESS }}/${K8S_API_SERVER}/g" /home/alpine/.kube/config 12 - sed -i "s/{{ KUBE_API_CA }}/${K8S_API_SERVER_CERT}/g" /home/alpine/.kube/config
设置Kubernetes的config文件,kube-config-template文件内容:
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
1 apiVersion: v1 2 clusters: 3 - cluster: 4 certificate-authority-data: {{ KUBE_API_CA }} 5 server: https://{{ KUBE_API_ADDRESS }} 6 name: helm 7 contexts: 8 - context: 9 cluster: helm 10 namespace: meshop-dev 11 user: helm 12 name: helm 13 current-context: "helm" 14 kind: Config 15 preferences: {} 16 users: 17 - name: helm 18 user: 19 token: {{ HELM_USER_TOKEN }}
获取KUBE_API_CA和HELM_USER_TOKEN的值
#kubectl get sa -A #查看所有security account #kubectl get sa/helm -n kube-system -o yaml #查看在某一个命名空间下的某一个security account=helm的yaml,得到secrets的name #kubectl get secrets/helm-token-wstgs -o yaml -n kube-system #查看在某一个命名空间下的某一个secrets的name的yaml,得到token值和ca.crt值
使用变量
设置默认分支:
设置仓库不允许推送代码,只能自己创建分支提交,在服务器上合并分支
git流水线克隆代码
流水线触发器的设置,添加一个token
设置sourceTree自定义操作,设置token=上面的值
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
1 curl -X POST -F token=f21cbb2d6f8c10cad915a380c49b99 -F 'ref=dev' -F 'variables[MESHOP_BUILD_ONLY]=sso' https://git.runshopstore.com/api/v4/projects/2/trigger/pipeline
设置ci里面使用触发器
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
1 workflow: 2 rules: 3 - if: $MESHOP_BUILD_ONLY != null 4 when: always 5 - if: $CI_PIPELINE_SOURCE =~ /^trigger|pipeline|web|api$/ 6 when: always
合并请求必须保证编译通过:同时删除合并请求的分支
设置关闭仓库,测试可以使用的项目:
设置标记:
规范化git commit信息
团队敏捷实践 —— 使用 semantic-release 自动管理发布版本
单个项目最简单的自动化脚本(Config项目)
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
stages: - restore - compile - version - build - deploy .load_environment_variables: &load_environment_variables - echo "init environment variables." - for environment in $(ls -d -1 .ci/environments/*.sh); do source $environment; done default: image: docker.tidebuy.net/dotnet/sdk:5.0 tags: - docker - company before_script: - *load_environment_variables variables: # docker in docker DOCKER_DRIVER: "overlay2" DOCKER_HOST: tcp://localhost:2375 # cache NUGET_PACKAGES_DIRECTORY: ".nuget" OBJECTS_DIRECTORY: "obj" SOURCE_CODE_PATH: "*/*/" .nuget_cache: cache: key: "${CI_PROJECT_NAME}-${CI_COMMIT_REF_SLUG}" paths: - "$SOURCE_CODE_PATH$OBJECTS_DIRECTORY/project.assets.json" - "$SOURCE_CODE_PATH$OBJECTS_DIRECTORY/*.csproj.nuget.*" - "$NUGET_PACKAGES_DIRECTORY" restore: stage: restore extends: .nuget_cache cache: policy: pull-push script: - echo "dotnet restore..." - dotnet restore $CI_PROJECT_DIR/MeShop.Config/MeShop.Config.sln --packages $NUGET_PACKAGES_DIRECTORY --runtime linux-x64 --configfile $CI_PROJECT_DIR/.ci/nuget.config only: refs: - branches - merge_requests compile: stage: compile before_script: - *load_environment_variables extends: - .nuget_cache cache: policy: pull script: - echo "compile project." - dotnet publish MeShop.Config/MeShop.View.Config/MeShop.View.Config.csproj --no-restore --runtime linux-x64 -c Release -o ${CI_PROJECT_DIR}/publish artifacts: name: "${CI_JOB_NAME}-${CI_COMMIT_REF_NAME}" paths: - publish/ expire_in: 1 days only: refs: - branches - merge_requests version: stage: version image: meshop/semantic-release:17.1.1 script: - if [ $CI_COMMIT_REF_NAME == 'alpha' -o $CI_COMMIT_REF_NAME == 'master' ]; then semantic-release; fi - if [ ! -f ./.version ]; then echo "VERSION=${MESHOP_BUILD_VERSION:-$(date +%Y%m%d%H%M%S)}">.version; fi - cat .version artifacts: paths: - .version expire_in: 1 days needs: - job: compile artifacts: false rules: - if: "$CI_PIPELINE_SOURCE == 'merge_request_event'" when: never - if: "$CI_COMMIT_REF_NAME == 'dev'" when: on_success - if: "$CI_COMMIT_REF_NAME == 'alpha'" when: on_success - if: "$CI_COMMIT_REF_NAME == 'master'" when: on_success - if: "$CI_COMMIT_REF_NAME =~ /.*-bug-fix/" when: on_success build: stage: build image: "docker:stable" services: - docker:stable-dind variables: DOCKER_TLS_CERTDIR: "" script: - echo "Logging to GitLab Container Registry with CI credentials..." - echo "$MESHOP_BUILD_REGISTRY_PASSWORD" | docker login -u "$MESHOP_BUILD_REGISTRY_USER" --password-stdin "$MESHOP_BUILD_REGISTRY" - source ./.version - image_version="${VERSION}" - image_name="${MESHOP_BUILD_REGISTRY}/meshop/shop/config" - docker build -f .ci/Dockerfile --tag "$image_name:$image_version" . - docker push "$image_name:$image_version" - if [ $MESHOP_BUILD_LATEST = 'true' ]; then docker tag $image_name:$image_version $image_name:latest && docker push "$image_name:latest" ; fi needs: - job: version artifacts: true - job: compile artifacts: true rules: - if: "$CI_PIPELINE_SOURCE == 'merge_request_event'" when: never - if: "$CI_COMMIT_REF_NAME == 'dev'" when: on_success - if: "$CI_COMMIT_REF_NAME == 'alpha'" when: on_success - if: "$CI_COMMIT_REF_NAME == 'master'" when: on_success - if: "$CI_COMMIT_REF_NAME =~ /.*-bug-fix/" when: on_success deploy: stage: deploy image: boxboat/kubectl:1.17.4 script: - mkdir -p /home/alpine/.kube - cp .ci/kube-config-template /home/alpine/.kube/config - sed -i "s/{{ HELM_USER_TOKEN }}/${HELM_USER_TOKEN}/g" /home/alpine/.kube/config - sed -i "s/{{ KUBE_API_ADDRESS }}/${KUBE_API_ADDRESS}/g" /home/alpine/.kube/config - sed -i "s/{{ KUBE_API_CA }}/${KUBE_API_CA}/g" /home/alpine/.kube/config - kubectl rollout restart deployment/meshop-config rules: - if: "$CI_PIPELINE_SOURCE == 'merge_request_event'" when: never - if: "$CI_COMMIT_REF_NAME == 'dev'" when: on_success