spotahome-redis-operator部署测试
环境:
k8s master:
q13756v
k8s node:
stark11
stark12
stark13
operator安装方法
git clone https://github.com/OT-CONTAINER-KIT/redis-operator
cd redis-operator/
helm upgrade redis-operator ./helm/redis-operator --install --namespace redis-operator
查看结果:
helm list -A | grep redis-operator
部署测试实例
修改helm模板
修改 helm/redis-setup/templates/redis-setup.yaml 增加redisConfig字段的支持
---
apiVersion: redis.opstreelabs.in/v1alpha1
kind: Redis
metadata:
name: {{ .Values.name }}
labels:
app.kubernetes.io/name: {{ .Values.name }}
helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/version: {{ .Chart.AppVersion }}
app.kubernetes.io/component: middleware
spec:
mode: {{ .Values.setupMode }}
{{- if .Values.cluster.size }}
size: {{ .Values.cluster.size }}
{{- end }}
global:
image: "{{ .Values.global.image }}:{{ .Values.global.tag }}"
imagePullPolicy: "{{ .Values.global.imagePullPolicy }}"
{{- if .Values.global.password }}
password: {{ .Values.global.password | quote }}
{{- end }}
{{- if .Values.global.redisConfig }}
redisConfig:
{{ toYaml .Values.global.redisConfig | indent 4 }}
{{- end }}
{{- if .Values.global.resources }}
resources:
{{ toYaml .Values.global.resources | indent 6 }}
{{- end }}
{{- if .Values.cluster.master }}
master:
service:
type: {{ .Values.cluster.master.serviceType }}
{{- end }}
{{- if .Values.cluster.slave }}
slave:
service:
type: {{ .Values.cluster.slave.serviceType }}
{{- end }}
redisExporter:
enabled: {{ .Values.exporter.enabled }}
image: "{{ .Values.exporter.image }}:{{ .Values.exporter.tag }}"
imagePullPolicy: {{ .Values.exporter.imagePullPolicy }}
{{- if .Values.exporter.resources }}
resources:
{{ toYaml .Values.exporter.resources | indent 6 }}
{{- end }}
{{- if .Values.storageSpec }}
storage:
{{ toYaml .Values.storageSpec | indent 4 }}
{{- end }}
{{- if .Values.priorityClassName }}
priorityClassName: {{ .Values.priorityClassName }}
{{- end }}
{{- if .Values.nodeSelector }}
nodeSelector:
{{ toYaml .Values.nodeSelector | indent 4 }}
{{- end }}
{{- if .Values.affinity }}
affinity:
{{ toYaml .Values.affinity | indent 4 }}
{{- end }}
{{- if .Values.securityContext }}
securityContext:
{{ toYaml .Values.securityContext | indent 4 }}
{{- end }}
集群模式
修改./helm/redis-setup/cluster-values.yaml默认参数为:
---
name: redis-cluster
setupMode: cluster
cluster:
size: 3
master:
serviceType: ClusterIP
slave:
serviceType: ClusterIP
global:
image: quay.io/opstree/redis
tag: "2.0"
imagePullPolicy: IfNotPresent
password: "Opstree@1234"
redisConfig:
timeout: "0"
tcp-keepalive: "300"
resources:
requests:
cpu: "10"
memory: 20Gi
limits:
cpu: "10"
memory: 20Gi
exporter:
enabled: true
image: quay.io/opstree/redis-exporter
tag: "1.0"
imagePullPolicy: IfNotPresent
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 100m
memory: 128Mi
# priorityClassName: "-"
nodeSelector: {}
# memory: medium
storageSpec:
volumeClaimTemplate:
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 20Gi
selector: {}
securityContext: {}
# runAsUser: 1000
affinity: {}
# nodeAffinity:
# requiredDuringSchedulingIgnoredDuringExecution:
# nodeSelectorTerms:
# - matchExpressions:
# - key: disktype
# operator: In
# values:
# - ssd
根据实际需求,创建1个redis cluster:
REDIS_CLUSTER_NAME=redis-cluster
helm upgrade ${REDIS_CLUSTER_NAME} ./helm/redis-setup -f ./helm/redis-setup/cluster-values.yaml --set name=${REDIS_CLUSTER_NAME} --set setupMode="cluster" --set cluster.size=3 --install --namespace redis-operator
单实例模式
修改./helm/redis-setup/standalone-values.yaml默认参数为:
---
name: redis
setupMode: standalone
cluster: {}
global:
image: quay.io/opstree/redis
tag: "2.0"
imagePullPolicy: IfNotPresent
password: "Opstree@1234"
redisConfig:
timeout: "0"
tcp-keepalive: "300"
resources:
requests:
cpu: "10"
memory: 20Gi
limits:
cpu: "10"
memory: 20Gi
exporter:
enabled: true
image: quay.io/opstree/redis-exporter
tag: "1.0"
imagePullPolicy: IfNotPresent
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 100m
memory: 128Mi
# priorityClassName: "-"
nodeSelector: {}
# memory: medium
storageSpec:
volumeClaimTemplate:
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 20Gi
selector: {}
securityContext: {}
# runAsUser: 1000
affinity: {}
# nodeAffinity:
# requiredDuringSchedulingIgnoredDuringExecution:
# nodeSelectorTerms:
# - matchExpressions:
# - key: disktype
# operator: In
# values:
# - ssd
根据实际需求,创建1个redis实例:
REDIS_NAME=redis
helm upgrade ${REDIS_NAME} ./helm/redis-setup -f ./helm/redis-setup/standalone-values.yaml --set name=${REDIS_NAME} --set setupMode="standalone" --install --namespace redis-operator
访问验证
在master节点上:
q13756v.add.bjyt.qihoo.net
## 查看自定义资源实例
kubectl get redis -A
## 进入客户端:
kubectl exec -it $(kubectl get po | grep centos7 | grep Running | awk '{print $1}') bash
(安装redis,以便使用客户端: yum install -y redis)
### 连接redis
#### 默认
redis-cli -h redis-cluster-master.redis-operator.svc -a Opstree@12345 -c
## auth "Opstree@1234"
set name "xiaoming"
exit
redis-cli -h redis-cluster-slave.redis-operator.svc -a Opstree@12345 -c
## auth "Opstree@1234"
get name
exit
#### nodeport(参考下面步骤配置nodeport)
redis-cli -h rg1-ceph101 -a Opstree@1234 -p 30379 -c
get name
exit
reids外部访问NodePort模式
因operator不支持其他类型service,需手工创建NodePort
apiVersion: v1
kind: Service
metadata:
namespace: redis-operator
name: nodeport-redis-cluster-master
annotations:
prometheus.io/port: "9121"
prometheus.io/scrape: "true"
redis.opstreelabs.in: "true"
labels:
app: redis-cluster-master
role: master
spec:
ports:
- name: redis-cluster-master
port: 6379
protocol: TCP
targetPort: 6379
nodePort: 30379
- name: redis-exporter
port: 9121
protocol: TCP
targetPort: 9121
nodePort: 30121
selector:
app: redis-cluster-master
role: master
sessionAffinity: None
type: NodePort
---
apiVersion: v1
kind: Service
metadata:
namespace: redis-operator
name: nodeport-redis-cluster-slave
annotations:
prometheus.io/port: "9121"
prometheus.io/scrape: "true"
redis.opstreelabs.in: "true"
labels:
app: redis-cluster-slave
role: slave
spec:
ports:
- name: redis-cluster-slave
port: 6379
protocol: TCP
targetPort: 6379
nodePort: 30479
- name: redis-exporter
port: 9121
protocol: TCP
targetPort: 9121
nodePort: 30221
selector:
app: redis-cluster-slave
role: slave
sessionAffinity: None
type: NodePort
---
apiVersion: v1
kind: Service
metadata:
annotations:
prometheus.io/port: "9121"
prometheus.io/scrape: "true"
redis.opstreelabs.in: "true"
labels:
app: redis-standalone
role: standalone
name: redis
namespace: redis-operator
spec:
ports:
- name: redis-standalone
port: 6379
protocol: TCP
targetPort: 6379
nodePort: 30579
- name: redis-exporter
port: 9121
protocol: TCP
targetPort: 9121
nodePort: 30321
selector:
app: redis-standalone
role: standalone
sessionAffinity: None
type: NodePort
清理
注意: 如果修改了存储配额,需要执行2-3步骤,否则直接跳到4:
1. 删除原来的服务(如果修改了存储配额):
helm uninstall ${REDIS_CLUSTER_NAME} -n redis-operator
2. 删除原来的存储(如果修改了存储配额):
kubectl get pvc -n redis-operator --no-headers | awk '{print $1}'
确认没问题删除:
kubectl get pvc -n redis-operator --no-headers | awk '{print $1}' | xargs kubectl delete pvc -n redis-operator