Kubernetes系列-部署Redis哨兵（k8s）

Kubernetes系列-部署Redis哨兵

需要准备自行准备k8s集群在此就不过多介绍，有需要的朋友可以查看我别的博客。

当然redis的哨兵配置有很多也可以不自己制造镜像，这里为了方便我们自己做的镜像实现起来会比较容易

一、准备配置文件

Redis配置文件

[root@master1 redis]# cat redis.conf 
bind 0.0.0.0
daemonize yes
logfile "" 
masterauth 123456
requirepass 123456
appendonly yes

Sentinel 配置文件

[root@master1 redis]# cat sentinel.conf 
bind 0.0.0.0
protected-mode no
port 26379
logfile "" 
sentinel monitor mymaster redis-0.redis 6379 2
sentinel auth-pass mymaster 123456
sentinel down-after-milliseconds mymaster 5000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 10000

二、准备容器执行脚本

[root@master1 redis]# cat redis.sh 
#!/bin/bash
pod_seq=`echo $POD_NAME | awk -F"-" '{print $2}'`
if [ $pod_seq -eq 0 ];then
    ip=`hostname -I`
    hostname -I > /usr/local/etc/redis/ip.txt
    sed -ri "/^bind/c bind $ip" /usr/local/etc/redis/redis.conf
    sed -ri "/^bind/c bind $ip" /usr/local/etc/redis/sentinel.conf
    sed -ri "/2$/c sentinel monitor mymaster $ip 6379 2" /usr/local/etc/redis/sentinel.conf
    redis-server /usr/local/etc/redis/redis.conf
    redis-sentinel /usr/local/etc/redis/sentinel.conf
else
    ip=`hostname -I`
    i=`cat /usr/local/etc/redis/ip.txt`
    sed -ri "/^bind/c bind $ip" /usr/local/etc/redis/redis.conf
    sed -ri "/^bind/c bind $ip" /usr/local/etc/redis/sentinel.conf
    sed -ri "/2$/c sentinel monitor mymaster $i 6379 2" /usr/local/etc/redis/sentinel.conf
    echo "replicaof redis-0.redis 6379" >> /usr/local/etc/redis/redis.conf
    redis-server /usr/local/etc/redis/redis.conf
    redis-sentinel /usr/local/etc/redis/sentinel.conf
fi

三、编写initContainer的Dockerfile

[root@master1 redis]# cat Dockerfile 
FROM alpine
COPY ["redis.conf","redis.sh","sentinel.conf","/redis/"]
构建并上传到自己的仓库

四、编写yaml文件

[root@master1 yaml]# cat redis.yaml 
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis
spec:
  serviceName: redis
  replicas: 3
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      initContainers:
      - image: registry.cn-shenzhen.aliyuncs.com/jun-lin/redis-cluser:v25
        name: conf
        command: ["cp","/redis/redis.sh","/redis/redis.conf","/redis/sentinel.conf","/conf"]
        volumeMounts:
        - mountPath: /conf
          name: redis-conf
      containers:
      - name: redis
        image: redis
        volumeMounts:
        - mountPath: /usr/local/etc/redis
          name: redis-conf
        command: ["sh","/usr/local/etc/redis/redis.sh"]
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        ports:
        - containerPort: 6379
          name: redis
      volumes:
      - name: redis-conf
        nfs:
          server: 192.168.17.129
          path: "/redis"

---
apiVersion: v1
kind: Service
metadata:
  name: redis
  labels:
    app: redis
spec:
  selector:
    app: redis
  clusterIP: None                  # Headless Service
  ports:
  - name: redis
    port: 6379

五、使用k8s按yaml文件进行创建

[root@master1 yaml]# kubectl apply -f redis.yaml 
statefulset.apps/redis created
service/redis created

[root@master1 yaml]# kubectl get po
NAME      READY   STATUS    RESTARTS   AGE
redis-0   1/1     Running   0          42m
redis-1   1/1     Running   0          42m
redis-2   1/1     Running   0          42m

六、进入容器内进行验证

1、验证集群是否正常

[root@master1 yaml]# kubectl exec -it redis-0 bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
root@redis-0:/data# redis-cli -h redis-0.redis -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
redis-0.redis:6379> INFO replication
# Replication
role:master
connected_slaves:2
slave0:ip=10.244.3.84,port=6379,state=online,offset=529387,lag=0
slave1:ip=10.244.3.83,port=6379,state=online,offset=529387,lag=0
master_replid:90befc6948f5a93401e73ab2acd5471e8f1e0d7d
master_replid2:722eb131e147ebd5bc57a5eee4e4f87cecebb684
master_repl_offset:529387
second_repl_offset:25163
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:529387
redis-1.redis:6379>

集群正常

2、验证哨兵是否正常

重启master上的redis

root@redis-0:/data# cat /run/redis.pid 
52
root@redis-0:/data# kill 52
root@redis-0:/data# redis-server /usr/local/etc/redis/redis.conf

重新查看集群情况

root@redis-0:/data# redis-cli -h redis-0.redis -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
redis-0.redis:6379> INFO repliction
redis-0.redis:6379> INFO replication
# Replication
role:slave
master_host:10.244.3.83
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:598226
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:90befc6948f5a93401e73ab2acd5471e8f1e0d7d
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:598226
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:58153
repl_backlog_histlen:540074
redis-0.redis:6379>

很显然master变成slave，查看redis-1上的情况

root@redis-0:/data# redis-cli -h redis-1.redis -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
redis-1.redis:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=10.244.3.84,port=6379,state=online,offset=615600,lag=1
slave1:ip=10.244.3.82,port=6379,state=online,offset=615600,lag=1
master_replid:90befc6948f5a93401e73ab2acd5471e8f1e0d7d
master_replid2:722eb131e147ebd5bc57a5eee4e4f87cecebb684
master_repl_offset:615600
second_repl_offset:25163
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:615600
redis-1.redis:6379>

集群正常