k8s operator开发与使用

k8s operator开发与使用

　　本篇文章用于记录k8s operator的开发，此处以一个简单的demo来记录operator的开发过程及相关要点；本文中使用kubebuilder工具来生成相关代码。

       实现一个daemonset资源对象的operator，作用为在每个节点启动一个nginx pod。

一、创建项目及初始化

#新建demo-operator的目录
[root@k8s-node4 data]# mkdir demo-operator
[root@k8s-node4 data]# cd demo-operator/

#Create a kubebuilder project, which requires an empty folder
[root@k8s-node4 demo-operator]# go mod init demo-operator
go: creating new go.mod: module demo-operator
[root@k8s-node4 demo-operator]# kubebuilder init --domain github.io
Writing kustomize manifests for you to edit...
Writing scaffold for you to edit...
Get controller runtime:
$ go get sigs.k8s.io/controller-runtime@v0.14.1
Update dependencies:
$ go mod tidy
Next: define a resource with:
$ kubebuilder create api

 #已经生成了一些初始化目录何文件

 [root@k8s-node4 demo-operator]# ll                
 total 96
 drwx------ 6 root root 4096 Feb 25 20:23 config
 -rw------- 1 root root 1250 Feb 25 20:23 Dockerfile
 -rw-r--r-- 1 root root 2919 Feb 25 20:23 go.mod
 -rw-r--r-- 1 root root 60004 Feb 25 20:23 go.sum
 drwx------ 2 root root 4096 Feb 25 20:23 hack
 -rw------- 1 root root 3483 Feb 25 20:23 main.go
 -rw------- 1 root root 7184 Feb 25 20:23 Makefile
 -rw------- 1 root root 326 Feb 25 20:23 PROJECT
 -rw------- 1 root root 2732 Feb 25 20:23 README.md

 二、创建API

[root@k8s-node4 demo-operator]# kubebuilder create api --group apps --version v1beta1 --kind MyDaemonset
Create Resource [y/n]
y
Create Controller [y/n]
y
Writing kustomize manifests for you to edit...
Writing scaffold for you to edit...
api/v1beta1/mydaemonset_types.go
controllers/mydaemonset_controller.go
Update dependencies:
$ go mod tidy
Running make:
$ make generate
mkdir -p /data/demo-operator/bin
test -s /data/demo-operator/bin/controller-gen && /data/demo-operator/bin/controller-gen --version | grep -q v0.11.1 || \
GOBIN=/data/demo-operator/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.11.1
/data/demo-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
Next: implement your new API and generate the manifests (e.g. CRDs,CRs) with:
$ make manifests

三、自定义scheme模板

      此处我们将模板的spec中添加image字段，status中添加avaliableReplicas字段

 

四、在controllers/mydaemonset_controller.go中添加逻辑

　　controller工作时，是一个生产者消费者模型，Reconcile为消费类型的方法，而SetupWithManager为生产者模型的方法，默认监听我们创建我们创建的

&appsv1beta1.MyDaemonset{}的动态，若有变更则会调用Reconcile的方法

五、Generate crd

#生成crd等资源
[root@k8s-node4 demo-operator]# make manifests
test -s /data/demo-operator/bin/controller-gen && /data/demo-operator/bin/controller-gen --version | grep -q v0.11.1 || \
GOBIN=/data/demo-operator/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.11.1
/data/demo-operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases

       由输出可见，crd资源放在当前目录的config/crd/bases目录下，config/crd/bases/apps.github.io_mydaemonsets.yaml 文件中会记录生产的crd信息。

六、make build & install

#make build 打包
[root@k8s-node4 demo-operator]# make build
test -s /data/demo-operator/bin/controller-gen && /data/demo-operator/bin/controller-gen --version | grep -q v0.11.1 || \
GOBIN=/data/demo-operator/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.11.1
/data/demo-operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/data/demo-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
api/v1beta1/mydaemonset_types.go
controllers/mydaemonset_controller.go
go vet ./...
go build -o bin/manager main.go

#make install 将crd资源部署到k8s中
[root@k8s-node4 demo-operator]# make install 
test -s /data/demo-operator/bin/controller-gen && /data/demo-operator/bin/controller-gen --version | grep -q v0.11.1 || \
GOBIN=/data/demo-operator/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.11.1
/data/demo-operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
test -s /data/demo-operator/bin/kustomize || { curl -Ss "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash -s -- 3.8.7 /data/demo-operator/bin; }
{Version:kustomize/v3.8.7 GitCommit:ad092cc7a91c07fdf63a2e4b7f13fa588a39af4f BuildDate:2020-11-11T23:14:14Z GoOs:linux GoArch:amd64}
kustomize installed to /data/demo-operator/bin/kustomize
/data/demo-operator/bin/kustomize build config/crd | kubectl apply -f -
customresourcedefinition.apiextensions.k8s.io/mydaemonsets.apps.github.io created

#此处可以发现crd已经创建成功

　[root@k8s-node4 demo-operator]# kubectl get customresourcedefinition.apiextensions.k8s.io/mydaemonsets.apps.github.io　　
  NAME CREATED AT
  mydaemonsets.apps.github.io 2023-02-25T13:07:46Z

七、make run

 

 　　根据输出可发现，程序已经启动成功；

八、测试是否能够成功运行pod

　　编辑config/samples/apps_v1beta1_mydaemonset.yaml文件，添加image字段，此字段为最开始我们定义scheme时的字段

 

       根据此文件创建pod

[root@k8s-node4 samples]# kubectl apply -f apps_v1beta1_mydaemonset.yaml 
mydaemonset.apps.github.io/mydaemonset-sample created

       此时我们查看pod信息，会发现在每一个node节点中，都运行了一个nginx的pod；

 

      并且我们可以在crd窗口看到，日志中添加的打印的输出信息；