k8s operator开发与使用
k8s operator开发与使用
本篇文章用于记录k8s operator的开发，此处以一个简单的demo来记录operator的开发过程及相关要点；本文中使用kubebuilder工具来生成相关代码。
实现一个daemonset资源对象的operator，作用为在每个节点启动一个nginx pod。
一、创建项目及初始化
#新建demo-operator的目录 [root@k8s-node4 data]# mkdir demo-operator [root@k8s-node4 data]# cd demo-operator/ #Create a kubebuilder project, which requires an empty folder [root@k8s-node4 demo-operator]# go mod init demo-operator go: creating new go.mod: module demo-operator [root@k8s-node4 demo-operator]# kubebuilder init --domain github.io Writing kustomize manifests for you to edit... Writing scaffold for you to edit... Get controller runtime: $ go get sigs.k8s.io/controller-runtime@v0.14.1 Update dependencies: $ go mod tidy Next: define a resource with: $ kubebuilder create api
#已经生成了一些初始化目录何文件
[root@k8s-node4 demo-operator]# ll
total 96
drwx------ 6 root root 4096 Feb 25 20:23 config
-rw------- 1 root root 1250 Feb 25 20:23 Dockerfile
-rw-r--r-- 1 root root 2919 Feb 25 20:23 go.mod
-rw-r--r-- 1 root root 60004 Feb 25 20:23 go.sum
drwx------ 2 root root 4096 Feb 25 20:23 hack
-rw------- 1 root root 3483 Feb 25 20:23 main.go
-rw------- 1 root root 7184 Feb 25 20:23 Makefile
-rw------- 1 root root 326 Feb 25 20:23 PROJECT
-rw------- 1 root root 2732 Feb 25 20:23 README.md
二、创建API
[root@k8s-node4 demo-operator]# kubebuilder create api --group apps --version v1beta1 --kind MyDaemonset Create Resource [y/n] y Create Controller [y/n] y Writing kustomize manifests for you to edit... Writing scaffold for you to edit... api/v1beta1/mydaemonset_types.go controllers/mydaemonset_controller.go Update dependencies: $ go mod tidy Running make: $ make generate mkdir -p /data/demo-operator/bin test -s /data/demo-operator/bin/controller-gen && /data/demo-operator/bin/controller-gen --version | grep -q v0.11.1 || \ GOBIN=/data/demo-operator/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.11.1 /data/demo-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..." Next: implement your new API and generate the manifests (e.g. CRDs,CRs) with: $ make manifests
三、自定义scheme模板
此处我们将模板的spec中添加image字段，status中添加avaliableReplicas字段
四、在controllers/mydaemonset_controller.go中添加逻辑
controller工作时，是一个生产者消费者模型，Reconcile为消费类型的方法，而SetupWithManager为生产者模型的方法，默认监听我们创建我们创建的
五、Generate crd
#生成crd等资源
[root@k8s-node4 demo-operator]# make manifests test -s /data/demo-operator/bin/controller-gen && /data/demo-operator/bin/controller-gen --version | grep -q v0.11.1 || \ GOBIN=/data/demo-operator/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.11.1 /data/demo-operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
由输出可见，crd资源放在当前目录的config/crd/bases目录下，config/crd/bases/apps.github.io_mydaemonsets.yaml 文件中会记录生产的crd信息。
六、make build & install
#make build 打包 [root@k8s-node4 demo-operator]# make build test -s /data/demo-operator/bin/controller-gen && /data/demo-operator/bin/controller-gen --version | grep -q v0.11.1 || \ GOBIN=/data/demo-operator/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.11.1 /data/demo-operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases /data/demo-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..." go fmt ./... api/v1beta1/mydaemonset_types.go controllers/mydaemonset_controller.go go vet ./... go build -o bin/manager main.go #make install 将crd资源部署到k8s中 [root@k8s-node4 demo-operator]# make install test -s /data/demo-operator/bin/controller-gen && /data/demo-operator/bin/controller-gen --version | grep -q v0.11.1 || \ GOBIN=/data/demo-operator/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.11.1 /data/demo-operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases test -s /data/demo-operator/bin/kustomize || { curl -Ss "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash -s -- 3.8.7 /data/demo-operator/bin; } {Version:kustomize/v3.8.7 GitCommit:ad092cc7a91c07fdf63a2e4b7f13fa588a39af4f BuildDate:2020-11-11T23:14:14Z GoOs:linux GoArch:amd64} kustomize installed to /data/demo-operator/bin/kustomize /data/demo-operator/bin/kustomize build config/crd | kubectl apply -f - customresourcedefinition.apiextensions.k8s.io/mydaemonsets.apps.github.io created
#此处可以发现crd已经创建成功
[root@k8s-node4 demo-operator]# kubectl get customresourcedefinition.apiextensions.k8s.io/mydaemonsets.apps.github.io
NAME CREATED AT
mydaemonsets.apps.github.io 2023-02-25T13:07:46Z
七、make run
根据输出可发现，程序已经启动成功；
八、测试是否能够成功运行pod
编辑config/samples/apps_v1beta1_mydaemonset.yaml文件，添加image字段，此字段为最开始我们定义scheme时的字段
根据此文件创建pod
[root@k8s-node4 samples]# kubectl apply -f apps_v1beta1_mydaemonset.yaml
mydaemonset.apps.github.io/mydaemonset-sample created
此时我们查看pod信息，会发现在每一个node节点中，都运行了一个nginx的pod；
并且我们可以在crd窗口看到，日志中添加的打印的输出信息；
