Redis Cluster集群
一、环境说明
| 服务器IP地址 | 服务器版本 | Redis版本 | Redis-master | Redis-slave |
| 10.10.18.10 | centos7.5 | 5.0 | 6379 | 6380 |
| 10.10.18.11 | centos7.5 | 5.0 | 6379 | 6380 |
| 10.10.18.12 | centos7.5 | 5.0 | 6379 | 6380 |
搭建redis集群，建议至少准备3台服务器，共搭建6个节点，3个master,3个slave，并且要求3个master节点不能全部跑到同一台服务器上，保证节点安全，3台服务器的配置相同， 每个节点运行两个端口。第一列做主库，第二列做备库
每个Redis群集节点都需要打开两个TCP连接。用于为客户端提供服务的普通Redis TCP端口，例如6379，加上通过向数据端口添加10000获得的端口，因此示例中为16379。
默认数据存放结构图：
这个结构会随着机器的故障自动调整master的角色，有可能原来是slave的角色，会变成master。
三个slave可以都宕机，数据没有问题
Redis Cluster提供了一种运行Redis安装的方法，其中数据 在多个Redis节点之间自动分片。
Redis Cluster还在分区期间提供一定程度的可用性，实际上是在某些节点发生故障或无法通信时继续运行的能力。但是，如果发生较大的故障（例如，当大多数主设备不可用时），群集将停止运行。
二、系统环境配置
系统参数配置
修改最大可打开文件数
修改文件/etc/security/limits.conf
* soft nofile 102400 * hard nofile 102400
TCP监听队列大小
文件/etc/sysctl.conf中增加一行
net.core.somaxconn = 32767
vm.overcommit_memory=1
然后执行命令“sysctl -p”以生效。
三、安装部署
下载redis5.0以上的版本
http://download.redis.io/releases/redis-5.0.3.tar.gz
tar -xzvf redis-5.0.3.tar.gz –C /usr/local cd /usr/local/redis-5.0.3 yum -y install gcc make MALLOC=libc mkdir –p /data/redis_data/ make install PREFIX=/data/redis_data/ mkdir –p /data/redis_data/6379/ cp /usr/local/redis-5.0.3/redis.conf /data/redis_data/6379/
修改：/data/redis_data/6379/redis.conf
bind 10.10.18.10 protected-mode no port 6379 daemonize yes cluster-enabled yes cluster-config-file /data/redis_data/6379/node.conf #这个文件是自动生成的文件 cluster-node-timeout 5000 pidfile /data/redis_data/6379/redis.pid logfile "/data/redis_data/6379/redis.log" dir /data/redis_data/6379 tcp-backlog 511 timeout 0 tcp-keepalive 300 supervised no loglevel notice databases 16 always-show-logo yes save 900 1 save 300 10 save 60 10000 stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum yes dbfilename dump.rdb replica-serve-stale-data yes replica-read-only yes repl-diskless-sync no repl-diskless-sync-delay 5 repl-disable-tcp-nodelay no replica-priority 100 lazyfree-lazy-eviction no lazyfree-lazy-expire no lazyfree-lazy-server-del no replica-lazy-flush no appendonly yes appendfilename "appendonly.aof" appendfsync everysec no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb aof-load-truncated yes aof-use-rdb-preamble yes lua-time-limit 5000 slowlog-log-slower-than 10000 slowlog-max-len 128 latency-monitor-threshold 0 notify-keyspace-events "" hash-max-ziplist-entries 512 hash-max-ziplist-value 64 list-max-ziplist-size -2 list-compress-depth 0 set-max-intset-entries 512 zset-max-ziplist-entries 128 zset-max-ziplist-value 64 hll-sparse-max-bytes 3000 stream-node-max-bytes 4096 stream-node-max-entries 100 activerehashing yes client-output-buffer-limit normal 0 0 0 client-output-buffer-limit replica 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 hz 10 dynamic-hz yes aof-rewrite-incremental-fsync yes rdb-save-incremental-fsync yes
修改：/data/redis_data/6379/redis.conf
mkdir –p /data/redis_data/6380/ cp /data/redis_data/6379/redis.conf /data/redis_data/6380/ sed -i 's/6379/6380/g' /data/redis_data/6380/redis.conf
编写启动脚本 /data/redis_data/start-redis.sh
#!/bin/sh REDIS_HOME=/data/redis_data $REDIS_HOME/bin/redis-server $REDIS_HOME/6379/redis.conf $REDIS_HOME/bin/redis-server $REDIS_HOME/6380/redis.conf
chmod +x /data/redis_data/start-redis.sh
开启服务
查看各服务器的进程情况
[root@redis01]# netstat -anpt |grep redis tcp 0 0 10.10.18.12:16379 0.0.0.0:* LISTEN 4328/redis-server 1 tcp 0 0 10.10.18.12:16380 0.0.0.0:* LISTEN 4330/redis-server 1 tcp 0 0 10.10.18.12:6379 0.0.0.0:* LISTEN 4328/redis-server 1 tcp 0 0 10.10.18.12:6380 0.0.0.0:* LISTEN 4330/redis-server 1
创建启动集群脚本 /data/redis_data/redis-cluster.sh
#!/bin/bash /data/redis_data/bin/redis-cli --cluster create 10.10.18.10:6379 10.10.18.11:6379 10.10.18.12:6379 10.10.18.10:6380 10.10.18.11:6380 10.10.18.12:6380 --cluster-replicas 1 参数说明 --cluster create：表示创建redis集群 --cluster-replicas 1：表示为集群中的每一个主节点指定一个从节点，即一比一的复制。
三台服务器上都使用以上的安装步骤。
在任意一台服务器上执行，开启集群服务：
[root@redis01]# sh redis-cluster.sh
结果
1 >>> Performing hash slots allocation on 6 nodes... 2 Master[0] -> Slots 0 - 5460 3 Master[1] -> Slots 5461 - 10922 4 Master[2] -> Slots 10923 - 16383 5 Adding replica 10.10.18.11:6380 to 10.10.18.10:6379 6 Adding replica 10.10.18.10:6380 to 10.10.18.11:6379 7 Adding replica 10.10.18.12:6380 to 10.10.18.12:6379 8 >>> Trying to optimize slaves allocation for anti-affinity 9 [OK] Perfect anti-affinity obtained! 10 M: c81b9adc68d5373237e9eaf3e5809bef42c5471a 10.10.18.10:6379 11 slots:[0-5460] (5461 slots) master 12 M: c4a495f621576910d934fa04da0896b39c0adf7a 10.10.18.11:6379 13 slots:[5461-10922] (5462 slots) master 14 M: baa5b1e2a769da1e37e3a4179ca362c9a26698c8 10.10.18.12:6379 15 slots:[10923-16383] (5461 slots) master 16 S: 8d315d8cc7f42afc5ac641003714f5e285822bff 10.10.18.10:6380 17 replicates c4a495f621576910d934fa04da0896b39c0adf7a 18 S: a7096a413407e77431c0d852c5106860d6d3184a 10.10.18.11:6380 19 replicates baa5b1e2a769da1e37e3a4179ca362c9a26698c8 20 S: 7f0c0858330bf107276d27c10f3e9c600aefa8ad 10.10.18.12:6380 21 replicates c81b9adc68d5373237e9eaf3e5809bef42c5471a 22 Can I set the above configuration? (type 'yes' to accept): yes 这里输入yes 23 >>> Nodes configuration updated 24 >>> Assign a different config epoch to each node 25 >>> Sending CLUSTER MEET messages to join the cluster 26 Waiting for the cluster to join 27 .... 28 >>> Performing Cluster Check (using node 10.10.18.10:6379) 29 M: c81b9adc68d5373237e9eaf3e5809bef42c5471a 10.10.18.10:6379 30 slots:[0-5460] (5461 slots) master 31 1 additional replica(s) 32 M: c4a495f621576910d934fa04da0896b39c0adf7a 10.10.18.11:6379 33 slots:[5461-10922] (5462 slots) master 34 1 additional replica(s) 35 S: 8d315d8cc7f42afc5ac641003714f5e285822bff 10.10.18.10:6380 36 slots: (0 slots) slave 37 replicates c4a495f621576910d934fa04da0896b39c0adf7a 38 S: 7f0c0858330bf107276d27c10f3e9c600aefa8ad 10.10.18.12:6380 39 slots: (0 slots) slave 40 replicates c81b9adc68d5373237e9eaf3e5809bef42c5471a 41 S: a7096a413407e77431c0d852c5106860d6d3184a 10.10.18.11:6380 42 slots: (0 slots) slave 43 replicates baa5b1e2a769da1e37e3a4179ca362c9a26698c8 44 M: baa5b1e2a769da1e37e3a4179ca362c9a26698c8 10.10.18.12:6379 45 slots:[10923-16383] (5461 slots) master 46 1 additional replica(s) 47 [OK] All nodes agree about slots configuration. 48 >>> Check for open slots... 49 >>> Check slots coverage... 50 [OK] All 16384 slots covered.
查看各服务器redis的状态
[root@redis01]# netstat -anpt | grep redis
结果：
1 tcp 0 0 10.10.18.12:16379 0.0.0.0:* LISTEN 4328/redis-server 1 2 tcp 0 0 10.10.18.12:16380 0.0.0.0:* LISTEN 4330/redis-server 1 3 tcp 0 0 10.10.18.12:6379 0.0.0.0:* LISTEN 4328/redis-server 1 4 tcp 0 0 10.10.18.12:6380 0.0.0.0:* LISTEN 4330/redis-server 1 5 tcp 0 0 10.10.18.12:46543 10.10.18.11:16380 ESTABLISHED 4330/redis-server 1 6 tcp 0 0 10.10.18.12:16379 10.10.18.10:40956 ESTABLISHED 4328/redis-server 1 7 tcp 0 0 10.10.18.12:16380 10.10.18.11:40559 ESTABLISHED 4330/redis-server 1 8 tcp 0 0 10.10.18.12:45872 10.10.18.10:16379 ESTABLISHED 4330/redis-server 1 9 tcp 0 0 10.10.18.12:45755 10.10.18.10:6379 ESTABLISHED 4330/redis-server 1 10 tcp 0 0 10.10.18.12:16379 10.10.18.12:36457 ESTABLISHED 4328/redis-server 1 11 tcp 0 0 10.10.18.12:39163 10.10.18.10:16379 ESTABLISHED 4328/redis-server 1 12 tcp 0 0 10.10.18.12:43185 10.10.18.10:16380 ESTABLISHED 4328/redis-server 1 13 tcp 0 0 10.10.18.12:42254 10.10.18.10:16380 ESTABLISHED 4330/redis-server 1 14 tcp 0 0 10.10.18.12:35352 10.10.18.11:16379 ESTABLISHED 4330/redis-server 1 15 tcp 0 0 10.10.18.12:41761 10.10.18.11:16379 ESTABLISHED 4328/redis-server 1 16 tcp 0 0 10.10.18.12:36457 10.10.18.12:16379 ESTABLISHED 4330/redis-server 1 17 tcp 0 0 10.10.18.12:16380 10.10.18.11:37557 ESTABLISHED 4330/redis-server 1 18 tcp 0 0 10.10.18.12:6379 10.10.18.11:46722 ESTABLISHED 4328/redis-server 1 19 tcp 0 0 10.10.18.12:16380 10.10.18.10:39318 ESTABLISHED 4330/redis-server 1 20 tcp 0 0 10.10.18.12:16379 10.10.18.10:35113 ESTABLISHED 4328/redis-server 1 21 tcp 0 0 10.10.18.12:16380 10.10.18.10:40872 ESTABLISHED 4330/redis-server 1 22 tcp 0 0 10.10.18.12:16380 10.10.18.12:34377 ESTABLISHED 4330/redis-server 1 23 tcp 0 0 10.10.18.12:16379 10.10.18.11:37717 ESTABLISHED 4328/redis-server 1 24 tcp 0 0 10.10.18.12:43984 10.10.18.11:16380 ESTABLISHED 4328/redis-server 1 25 tcp 0 0 10.10.18.12:16379 10.10.18.11:45994 ESTABLISHED 4328/redis-server 1 26 tcp 0 0 10.10.18.12:34377 10.10.18.12:16380 ESTABLISHED 4328/redis-server 1
四、测试
设置redis密码
[root@redis01 bin]# /data/redis_data/bin/redis-cli -h 10.10.18.10 -p 6379 -c
结果：
10.10.18.10:6379> keys * (empty list or set) 10.10.18.10:6379> config set masterauth redis20190628 OK 10.10.18.10:6379> config set requirepass redis20190628 OK 10.10.18.10:6379> exit
查看所有的节点信息：
[root@redis01]# /data/redis_data/bin/redis-cli -h 10.10.18.10 -p 6380 -c -a 'redis20190628' cluster nodes
结果：
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. c4a495f621576910d934fa04da0896b39c0adf7a 10.10.18.11:6379@16379 master - 0 1551081545000 2 connected 5461-10922 c81b9adc68d5373237e9eaf3e5809bef42c5471a 10.10.18.10:6379@16379 master - 0 1551081544292 1 connected 0-5460 8d315d8cc7f42afc5ac641003714f5e285822bff 10.10.18.10:6380@16380 myself,slave c4a495f621576910d934fa04da0896b39c0adf7a 0 1551081543000 4 connected baa5b1e2a769da1e37e3a4179ca362c9a26698c8 10.10.18.12:6379@16379 master - 0 1551081543000 3 connected 10923-16383 7f0c0858330bf107276d27c10f3e9c600aefa8ad 10.10.18.12:6380@16380 slave c81b9adc68d5373237e9eaf3e5809bef42c5471a 0 1551081544292 6 connected a7096a413407e77431c0d852c5106860d6d3184a 10.10.18.11:6380@16380 slave baa5b1e2a769da1e37e3a4179ca362c9a26698c8 0 1551081543292 5 connected
添加测试数据
在任意一台机器上执行插入数据，然后在其它任意节点上都能获取到对应的值
[root@redis01]# /data/redis_data/bin/redis-cli -h 10.10.18.10 -p 6380 -c -a 'redis20190628' Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 10.10.18.10:6380> set test10_6380 test [root@redis01]# /data/redis_data/bin/redis-cli -h 10.10.18.10 -p 6379 -c -a 'redis20190628' Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 10.10.18.10:6379> get test10_6380 "test" [root@redis02]# /data/redis_data/bin/redis-cli -h 10.10.18.11 -p 6380 -c -a 'redis20190628' Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 10.10.18.11:6380> get test10_6380 -> Redirected to slot [2317] located at 10.10.18.10:6379 "test"
