新一代云原生CI平台-drone 学习总结

1 基本介绍

Drone is a self-service Continuous Integration platform for busy development teams.

drone是个ci平台，支持开发团队的自助使用。

drone与其他如jenkins等常用ci平台的优点主要在：轻量易用、扩展方便、云原生等，本身用go语言编写，实现很优雅，值得学习。

1.1 云原生

流程中所有的任务执行都基于容器

1.2 server

server, drone 的平台入口，为drone-ui、drone-cli提供http api实现，额外功能包括：ssl认证、cookie管理、log打印、metric查询接口（可对接Prometheus）。

存储

存储默认使用嵌入式数据库sqlite，可支持mysql（5.6+）、postgres（推荐）。

server默认会将大的text files存在数据库，现只支持amazon s3作为可选替代，后续需进行调整到cos或自建对象存储。

用户管理

现支持管理员、普通成员两中角色，管理员拥有所有权限。另外支持非自然人“机器用户”，用来做自动化和集成。

drone可与github的team团队管理角色打通，可自动对应角色和权限。

1.3 runner

runner 就是轮询拉取流水线进行执行的独立守护进程。

A runner is a standalone daemon that polls the server for pending pipelines to execute.

There are different types of runners optimized for different use cases and runtime environments. You can install one or many runners, of one or many types.

目前主要使用的是 docker runner。

docker runner：The Docker runner is a daemon that executes pipelines steps inside ephemeral Docker containers. 在容器中执行流水线steps。

kubernetes runner： The Kubernetes runner is a standalone service that executes pipelines inside Pods. 在k8s pod执行流水线。

1.4 piplines

通过编排，流水线帮助我们将软件交付流程自动化。

Pipelines help you automate steps in your software delivery process, such as initiating code builds, running automated tests, and deploying to a staging or production environment.

当前主要使用 docker和k8s 两种type：

Docker pipelines execute pipeline commands inside ephemeral Docker containers. Docker containers provide isolation, allowing safe execution of concurrent pipelines on the same machine

A kubernetes pipeline executes pipeline steps as containers inside a Kubernetes Pod. Containers provide isolation allowing safe execution of concurrent pipelines on the same machine.

1.5 基本的使用流程

2 功能使用

2.1 pipline as code

drone没有提供可视化页面进行拖拖拽拽完成流水线的搭建，直接as code化，通过项目中的.drone.yaml文件完成整个流水线的配置。

这里给一个尽量包含所有特性的例子，仅供参考了解。完整语法参见：

Configure | Drone

Docker Pipeline | Drone

kind: pipeline
type: docker # docker or k8s
name: linux-amd64

# The root of your git repository, also called the **workspace**, is shared by all steps in your pipeline

platform: # configure the target operating system and architecture and routes the pipeline to the appropriate runner
  arch: amd64
  os: linux

steps: # steps are defined as a series of shell commands
- name: test
  image: golang:1.14.15 # Each step must therefore define the Docker **image** used to create the container.
  # **Plugins** are docker containers that encapsulate commands, and can be shared and re-used in your pipeline.
  commands: # The commands are executed inside the root directory of your git repository.
  - go test ./...
  - go build -o /dev/null github.com/drone/drone/cmd/drone-server
  - go build -o /dev/null -tags "oss nolimit" github.com/drone/drone/cmd/drone-server
  
- name: build
  image: golang:1.14.15
  commands:
  - sh scripts/build.sh
  environment: # The environment section provides the ability to define environment variables scoped to individual pipeline steps.
    GOARCH: amd64
    GOOS: linux

- name: publish
  image: plugins/docker:18
  settings: # inputs for plugin as environment variables
    auto_tag: true
    auto_tag_suffix: linux-amd64
    dockerfile: docker/Dockerfile.server.linux.amd64
    repo: drone/drone
    username:
      from_secret: docker_username
    password:
      from_secret: docker_password
  when: # The when section provides the ability to conditionally limit the execution of steps at runtime.
    event: 
    - push
    - tag

trigger: # receive webhook from scm(gitlab/github), filter by bransh/ref/event
  event:
  - push
  - tag

2.2 其他的配置项

depends_on

通过depends_on声明依赖关系，可以将流水线构建为一个有向无环图，并支持并发。

流水线在不使用depends_on情况下，所有steps是顺序执行的。

kind: pipeline
type: docker
name: default

steps:
- name: backend
  image: golang
  commands:
  - go build
  - go test

- name: frontend
  image: node
  commands:
  - npm install
  - npm test

- name: notify
  image: plugins/slack
  settings:
    webhook:
      from_secret: webhook
  depends_on:
  - frontend
  - backend

volumes

mount host/temporary volumes to share state between pipline steps.

下面的实例通过volumes实现go test 和 go build两个step公用拉取的go 依赖包。

kind: pipeline
type: docker
name: default

steps:
- name: test
  image: golang
  volumes:
  - name: cache
    path: /go
  commands:
  - go get
  - go test

- name: build
  image: golang
  volumes:
  - name: cache
    path: /go
  commands:
  - go build

volumes:
- name: cache
  temp: {}

services

drone支持通过声明一些独立的service供流水线使用，如redis、mysql等单测流程中可使用的，独立的service会持续运行直到流水线结束，service的执行失败不会影响流水线整体状态。

service 在具体使用时会有一些常见问题，如错误的使用localhost或127.0.0.1来连接service；service运行在容器中，需要时间去初始化，具体使用时需要通过shell while 或sleep等方式或检测或等待service初始完成，具体参见：Services | Drone

Drone supports launching detached service containers as part of your pipeline.

kind: pipeline
type: docker
name: default

services:
- name: cache
  image: redis

2.3 Secrets

敏感信息管理，避免在配置中出现密码等敏感数据。

Repository/Organization secrets： are used to store and manage sensitive information, such as passwords, tokens, and ssh keys

kind: pipeline
name: default

steps:
- name: build
  image: alpine
  environment:
    USERNAME:
      from_secret: docker_username # 此变量由drone统一存储
    PASSWORD:
      from_secret: docker_password

2.4 Signatures

配置文件签名，防止非法pull request篡改配置文件，暂时可能用不上。

2.5 Promotions

主要为了提供流水线发布的功能，通过不同的event和参数与流水线平时的build区分开。

通过drone cli 发起promote，pipline可接收的相关event和target信息后进行发布

Create repeatable deployments：创建可复用的发布
Create an audit trail：记录审计日志
Reduce human error：较少人为失误
Segregation of duties：责任分离
Revoke developer access to server environments：取消开发者访问服务环境的权限

2.6 Cron

定时触发执行流水线。在repo的setting中进行配置，或者通过drone cli。

You can use cron jobs to execute pipelines on time-based schedules.

2.7 Templates

通过复用简化流水线配置复杂度，减少重复配置.

如：use template plugin.yaml

kind: template
load: plugin.yaml
data:
  name: name
  image: image
  commands: commands

template：基于 go template

kind: pipeline
type: docker
name: default
steps:
   - name: {{ .input.name }}
     image: {{ .input.image }}
     commands:
        - {{ .input.commands }}

3 插件支持

3.1 Plugins

基于容器，可用任何语言编写

Plugins are Docker containers that perform pre-defined tasks and are configured as steps in your pipeline.

workspace

drone会自动clone代码到默认工作空间，plugin无需自己检出代码

inputs

Plugins parameters are defined in the settings section of the pipeline step and are passed to the plugin container as environment variables.

- name: publish
  image: plugins/docker
  settings:
    username: kevinbacon
    password: pa55word
    repo: foo/bar
    tags:
    - 1.0.0
    - 1.0

插件进程拿到的环境变量如下：

PLUGIN_USERNAME=kevinbacon 
PLUGIN_PASSWORD=pa55word 
PLUGIN_REPO=foo/bar 
PLUGIN_TAGS=1.0.0,1.0

outputs

暂时只有成功和失败两种状态返回，即step的状态。

The container exit code is used to determine whether the step is passing or failing. If a command returns a non-zero exit code, the step is marked as failing. The overall pipeline status is also marked as failing, and remaining pipeline steps are skipped (unless explicitly configured to run on failure).

插件发布

Plugins are distributed as Docker images. You can publish plugins to any Docker registry, private or public, to share plugins internally with your organization, or publicly with the broader developer community.

已有插件库：https://plugins.drone.io/

Example Go Plugin: https://docs.drone.io/plugins/tutorials/golang/

3.2 Webhooks

流水线事件的对外分发。

System webhooks can be used to send an http request to a designated endpoint every time a system event occurs. Example system events:

User is created
User is deleted
Repository is activated
Repository is de-activated
Build is created
Build is updated or completed

example：

{ 
"action": "completed", 
"repo": { 
"id": 42, 
"uid": "16607898", 
"user_id": 2, 
"namespace": "octocat", 
"name": "octocat", 
"slug": "octocat/hello-world" 
... 
}

3.3 Extensions

对于drone不满足的功能，我们可以自建拓展程序，扩展就是简单的http service。

比如：添加自己账号体系的权限校验，drone.yaml文件的预检、补充steps，添加自定义环境变量等

3.4 Autoscaler

弹性伸缩。

The Drone Autoscaler is a standalone daemon that continuously polls your build queue and provisions or terminates server instances based on volume.

4 Open API

Drone provides a comprehensive remote API for interacting with the Drone server.

4.1 Libraries

github.com/drone/drone-go
github.com/drone/drone-js
github.com/drone/drone-node

4.2 Endpoints

Build Endpoint
Cron Endpoint
Repository Endpoint
Secrets Endpoint
User Endpoint
Users Endpoint

5 CLI

The Drone command line tools are used to interact with the Drone from the command line, and provide important utilities for managing users and repository settings.