为 Memcached 构建基于 Go 的 Operator 示例
Operator SDK 中的 Go 编程语言支持可以利用 Operator SDK 中的 Go 编程语言支持，为 Memcached 构
建基于 Go 的 Operator 示例、分布式键值存储并管理其生命周期。
github: https://github.com/jxlwqq/memcached-operator
前置条件
- 安装 Docker Desktop，并启动内置的 Kubernetes 集群
- 注册一个 hub.docker.com 账户，需要将本地构建好的镜像推送至公开仓库中
- 安装 operator SDK CLI:
brew install operator-sdk - 安装 Go:
brew install go
创建项目
使用 Operator SDK CLI 创建名为 memcached-operator 的项目。
mkdir -p $HOME/projects/memcached-operator
cd $HOME/projects/memcached-operator
go env -w GOPROXY=https://goproxy.cn,direct
operator-sdk init \
--domain=jxlwqq.github.io \
--repo=github.com/jxlwqq/memcached-operator \
--skip-go-version-check
创建 API 和控制器
使用 Operator SDK CLI 创建自定义资源定义（CRD）API 和控制器。
运行以下命令创建带有组 cache、版本 v1alpha1 和种类 Memcached 的 API：
operator-sdk create api \
--resource=true \
--controller=true \
--group=cache \
--version=v1alpha1 \
--kind=Memcached
定义 Memcached 自定义资源（CR）的 API。
修改 api/v1alpha1/memcached_types.go 中的 Go 类型定义，使其具有以下 spec 和 status
type MemcachedSpec struct {
// +kubebuilder:validation:Minimum=0
// Size is the size of the memcached deployment
Size int32 `json:"size"`
}
type MemcachedStatus struct {
// Nodes are the names of the memcached pods
Nodes []string `json:"nodes"`
}
为资源类型更新生成的代码：
make generate
运行以下命令以生成和更新 CRD 清单：
make manifests
实现控制器
在本例中，将生成的控制器文件 controllers/memcached_controller.go 替换为以下示例实现：
/*
Copyright 2021.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package controllers
import (
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
"reflect"
"time"
"context"
"k8s.io/apimachinery/pkg/runtime"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
ctrllog "sigs.k8s.io/controller-runtime/pkg/log"
cachev1alpha1 "github.com/jxlwqq/memcached-operator/api/v1alpha1"
)
// MemcachedReconciler reconciles a Memcached object
type MemcachedReconciler struct {
client.Client
Scheme *runtime.Scheme
}
//+kubebuilder:rbac:groups=cache.jxlwqq.github.io,resources=memcacheds,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=cache.jxlwqq.github.io,resources=memcacheds/status,verbs=get;update;patch
//+kubebuilder:rbac:groups=cache.jxlwqq.github.io,resources=memcacheds/finalizers,verbs=update
//+kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=core,resources=pods,verbs=get;list;watch
// Reconcile is part of the main kubernetes reconciliation loop which aims to
// move the current state of the cluster closer to the desired state.
// TODO(user): Modify the Reconcile function to compare the state specified by
// the Memcached object against the actual cluster state, and then
// perform operations to make the cluster state reflect the state specified by
// the user.
//
// For more details, check Reconcile and its Result here:
// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.9.2/pkg/reconcile
func (r *MemcachedReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
log := ctrllog.FromContext(ctx)
// Fetch the Memcached instance
memcached := &cachev1alpha1.Memcached{}
err := r.Get(ctx, req.NamespacedName, memcached)
if err != nil {
if errors.IsNotFound(err) {
// Request object not found, could have been deleted after reconcile request.
// Owned objects are automatically garbage collected. For additional cleanup logic use finalizers.
// Return and don't requeue
log.Info("Memcached resource not found. Ignoring since object must be deleted")
return ctrl.Result{}, nil
}
// Error reading the object - requeue the request.
log.Error(err, "Failed to get Memcached")
return ctrl.Result{}, err
}
// Check if the deployment already exists, if not create a new one
found := &appsv1.Deployment{}
err = r.Get(ctx, types.NamespacedName{Name: memcached.Name, Namespace: memcached.Namespace}, found)
if err != nil && errors.IsNotFound(err) {
// Define a new deployment
dep := r.deploymentForMemcached(memcached)
log.Info("Creating a new Deployment", "Deployment.Namespace", dep.Namespace, "Deployment.Name", dep.Name)
err = r.Create(ctx, dep)
if err != nil {
log.Error(err, "Failed to create new Deployment", "Deployment.Namespace", dep.Namespace, "Deployment.Name", dep.Name)
return ctrl.Result{}, err
}
// Deployment created successfully - return and requeue
return ctrl.Result{Requeue: true}, nil
} else if err != nil {
log.Error(err, "Failed to get Deployment")
return ctrl.Result{}, err
}
// Ensure the deployment size is the same as the spec
size := memcached.Spec.Size
if *found.Spec.Replicas != size {
found.Spec.Replicas = &size
err = r.Update(ctx, found)
if err != nil {
log.Error(err, "Failed to update Deployment", "Deployment.Namespace", found.Namespace, "Deployment.Name", found.Name)
return ctrl.Result{}, err
}
// Ask to requeue after 1 minute in order to give enough time for the
// pods be created on the cluster side and the operand be able
// to do the next update step accurately.
return ctrl.Result{RequeueAfter: time.Minute}, nil
}
// Update the Memcached status with the pod names
// List the pods for this memcached's deployment
podList := &corev1.PodList{}
listOpts := []client.ListOption{
client.InNamespace(memcached.Namespace),
client.MatchingLabels(labelsForMemcached(memcached.Name)),
}
if err = r.List(ctx, podList, listOpts...); err != nil {
log.Error(err, "Failed to list pods", "Memcached.Namespace", memcached.Namespace, "Memcached.Name", memcached.Name)
return ctrl.Result{}, err
}
podNames := getPodNames(podList.Items)
// Update status.Nodes if needed
if !reflect.DeepEqual(podNames, memcached.Status.Nodes) {
memcached.Status.Nodes = podNames
err := r.Status().Update(ctx, memcached)
if err != nil {
log.Error(err, "Failed to update Memcached status")
return ctrl.Result{}, err
}
}
return ctrl.Result{}, nil
}
// deploymentForMemcached returns a memcached Deployment object
func (r *MemcachedReconciler) deploymentForMemcached(m *cachev1alpha1.Memcached) *appsv1.Deployment {
ls := labelsForMemcached(m.Name)
replicas := m.Spec.Size
dep := &appsv1.Deployment{
ObjectMeta: metav1.ObjectMeta{
Name: m.Name,
Namespace: m.Namespace,
},
Spec: appsv1.DeploymentSpec{
Replicas: &replicas,
Selector: &metav1.LabelSelector{
MatchLabels: ls,
},
Template: corev1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: ls,
},
Spec: corev1.PodSpec{
Containers: []corev1.Container{{
Image: "memcached:1.4.36-alpine",
Name: "memcached",
Command: []string{"memcached", "-m=64", "-o", "modern", "-v"},
Ports: []corev1.ContainerPort{{
ContainerPort: 11211,
Name: "memcached",
}},
}},
},
},
},
}
// Set Memcached instance as the owner and controller
ctrl.SetControllerReference(m, dep, r.Scheme)
return dep
}
// labelsForMemcached returns the labels for selecting the resources
// belonging to the given memcached CR name.
func labelsForMemcached(name string) map[string]string {
return map[string]string{"app": "memcached", "memcached_cr": name}
}
// getPodNames returns the pod names of the array of pods passed in
func getPodNames(pods []corev1.Pod) []string {
var podNames []string
for _, pod := range pods {
podNames = append(podNames, pod.Name)
}
return podNames
}
// SetupWithManager sets up the controller with the Manager.
func (r *MemcachedReconciler) SetupWithManager(mgr ctrl.Manager) error {
return ctrl.NewControllerManagedBy(mgr).
For(&cachev1alpha1.Memcached{}).
Owns(&appsv1.Deployment{}).
Complete(r)
}
运行以下命令以生成和更新 CRD 清单：
make manifests
运行 Operator
捆绑 Operator，并使用 Operator Lifecycle Manager（OLM）在集群中部署。
修改 Makefile 中 IMAGE_TAG_BASE 和 IMG：
IMAGE_TAG_BASE ?= docker.io/jxlwqq/memcached-operator
IMG ?= $(IMAGE_TAG_BASE):latest
构建镜像：
make docker-build
将镜像推送到镜像仓库：
make docker-push
成功后访问：https://hub.docker.com/r/jxlwqq/memcached-operator
运行 make bundle 命令创建 Operator 捆绑包清单，并依次填入名称、作者等必要信息:
make bundle
构建捆绑包镜像：
make bundle-build
推送捆绑包镜像：
make bundle-push
成功后访问：https://hub.docker.com/r/jxlwqq/memcached-operator-bundle
使用 Operator Lifecycle Manager 部署 Operator:
# 切换至本地集群
kubectl config use-context docker-desktop
# 安装 olm
operator-sdk olm install
# 使用 Operator SDK 中的 OLM 集成在集群中运行 Operator
operator-sdk run bundle docker.io/jxlwqq/memcached-operator-bundle:v0.0.1
查看 Pod：
NAME READY STATUS RESTARTS AGE
7bb60cfcec4a426a75d6ca01153f5e6d3061e175d035791255372bdd7ebrvmd 0/1 Completed 0 67s
docker-io-jxlwqq-memcached-operator-bundle-v0-0-1 1/1 Running 0 84s
memcached-operator-controller-manager-666cfff6c-fxhrk 2/2 Running 0 41s
创建自定义资源
编辑 config/samples/cache_v1alpha1_memcached.yaml 上的 Memcached CR 清单示例，使其包含
以下规格：
apiVersion: cache.jxlwqq.github.io/v1alpha1
kind: Memcached
metadata:
name: memcached-sample
spec:
size: 2
创建 CR：
kubectl apply -f config/samples/cache_v1alpha1_memcached.yaml
查看 Pod：
NAME READY STATUS RESTARTS AGE
7bb60cfcec4a426a75d6ca01153f5e6d3061e175d035791255372bdd7ebrvmd 0/1 Completed 0 4m14s
docker-io-jxlwqq-memcached-operator-bundle-v0-0-1 1/1 Running 0 4m31s
memcached-operator-controller-manager-666cfff6c-fxhrk 2/2 Running 0 3m48s
memcached-sample-6c765df685-d8ppn 1/1 Running 0 29s
memcached-sample-6c765df685-dtw9l 1/1 Running 0 29s
更新 CR：
kubectl patch memcached memcached-sample -p '{"spec":{"size": 3}}' --type=merge
查看 Pod：
NAME READY STATUS RESTARTS AGE
7bb60cfcec4a426a75d6ca01153f5e6d3061e175d035791255372bdd7ebrvmd 0/1 Completed 0 6m52s
docker-io-jxlwqq-memcached-operator-bundle-v0-0-1 1/1 Running 0 7m9s
memcached-operator-controller-manager-666cfff6c-fxhrk 2/2 Running 0 6m26s
memcached-sample-6c765df685-d8ppn 1/1 Running 0 3m7s
memcached-sample-6c765df685-dtw9l 1/1 Running 0 3m7s
memcached-sample-6c765df685-n7ctq 1/1 Running 0 6s
做好清理
operator-sdk cleanup memcached-operator
operator-sdk olm uninstall
