1、基础
1.1、非集群的问题点
容量不够，redis如何进行扩容？
并发写操作， redis如何分摊？
另外，主从模式，主机宕机，导致ip地址发生变化，应用程序中配置需要修改对应的主机地址、端口等信息。
之前通过代理主机来解决，但是redis3.0中提供了解决方案。就是无中心化集群配置。
1.2、什么是redis集群
Redis 集群实现了对Redis的水平扩容，即启动N个redis节点，将整个数据库分布存储在这N个节点中，每个节点存储总数据的1/N。
Redis 集群通过分区（partition）来提供一定程度的可用性（availability）： 即使集群中有一部分节点失效或者无法进行通讯， 集群也可以继续处理命令请求。
2、环境准备
2.1、主机准备
redis-01 192.168.10.19
redis-02 192.168.10.20
2.2、实例规划
每台主机3个实例，端口分别：6379、6380、6381
3、配置集群
3.1、主配置文件redis.conf
]# cat /usr/local/redis-6.2.5/etc/redis.conf | grep -Ev '^$|^#'
bind *
port 6379
daemonize yes
...
3.2、redis-01配置
3.2.1、6379.conf
redis-01 ~]# cat /usr/local/redis-6.2.5/etc/6379.conf
include /usr/local/redis-6.2.5/etc/redis.conf
logfile "/usr/local/redis-6.2.5/log/6379.log"
dbfilename "dump_6379.rdb"
pidfile "/var/run/redis_6379.pid"
dir "/usr/local/redis-6.2.5/data"
daemonize yes
port 6379
cluster-enabled yes
cluster-config-file nodes-6379.conf
cluster-node-timeout 15000
3.2.2、6380.conf
redis-01 ~]# cat /usr/local/redis-6.2.5/etc/6380.conf
include /usr/local/redis-6.2.5/etc/redis.conf
logfile "/usr/local/redis-6.2.5/log/6380.log"
dbfilename "dump_6380.rdb"
pidfile "/var/run/redis_6380.pid"
dir "/usr/local/redis-6.2.5/data"
daemonize yes
port 6380
cluster-enabled yes
cluster-config-file nodes-6380.conf
cluster-node-timeout 15000
3.2.3、6381.conf
redis-01 ~]# cat /usr/local/redis-6.2.5/etc/6381.conf
include /usr/local/redis-6.2.5/etc/redis.conf
logfile "/usr/local/redis-6.2.5/log/6381.log"
dbfilename "dump_6381.rdb"
pidfile "/var/run/redis_6381.pid"
dir "/usr/local/redis-6.2.5/data"
daemonize yes
port 6381
cluster-enabled yes
cluster-config-file nodes-6381.conf
cluster-node-timeout 15000
3.3、redis-02配置
3.3.1、6379.conf
redis-02 ~]# cat /usr/local/redis-6.2.5/etc/6379.conf
include /usr/local/redis-6.2.5/etc/redis.conf
logfile "/usr/local/redis-6.2.5/log/6379.log"
dbfilename "dump_6379.rdb"
pidfile "/var/run/redis_6379.pid"
dir "/usr/local/redis-6.2.5/data"
daemonize yes
port 6379
cluster-enabled yes
cluster-config-file nodes-6379.conf
cluster-node-timeout 15000
3.3.2、6380.conf
redis-02 ~]# cat /usr/local/redis-6.2.5/etc/6380.conf
include /usr/local/redis-6.2.5/etc/redis.conf
logfile "/usr/local/redis-6.2.5/log/6380.log"
dbfilename "dump_6380.rdb"
pidfile "/var/run/redis_6380.pid"
dir "/usr/local/redis-6.2.5/data"
daemonize yes
port 6380
cluster-enabled yes
cluster-config-file nodes-6380.conf
cluster-node-timeout 15000
3.3.3、6381.conf
redis-02 ~]# cat /usr/local/redis-6.2.5/etc/6381.conf
include /usr/local/redis-6.2.5/etc/redis.conf
logfile "/usr/local/redis-6.2.5/log/6381.log"
dbfilename "dump_6381.rdb"
pidfile "/var/run/redis_6381.pid"
dir "/usr/local/redis-6.2.5/data"
daemonize yes
port 6381
cluster-enabled yes
cluster-config-file nodes-6381.conf
cluster-node-timeout 15000
3.4、启动redis服务、检查端口状态、查询配置文件生成
3.4.1、redis-01
# redis-01
redis-server /usr/local/redis-6.2.5/etc/6379.conf
redis-server /usr/local/redis-6.2.5/etc/6380.conf
redis-server /usr/local/redis-6.2.5/etc/6381.conf
redis-01 ~]# netstat -tunlp | grep redis
tcp 0 0 0.0.0.0:6379 0.0.0.0:* LISTEN 29696/redis-server
tcp 0 0 0.0.0.0:6380 0.0.0.0:* LISTEN 29702/redis-server
tcp 0 0 0.0.0.0:6381 0.0.0.0:* LISTEN 29708/redis-server
tcp 0 0 0.0.0.0:16379 0.0.0.0:* LISTEN 29696/redis-server
tcp 0 0 0.0.0.0:16380 0.0.0.0:* LISTEN 29702/redis-server
tcp 0 0 0.0.0.0:16381 0.0.0.0:* LISTEN 29708/redis-server
redis-01 ~]# ll /usr/local/redis-6.2.5/data/
总用量 12
-rw-r--r--. 1 root root 126 5月 16 20:52 nodes-6379.conf
-rw-r--r--. 1 root root 126 5月 16 20:52 nodes-6380.conf
-rw-r--r--. 1 root root 126 5月 16 20:52 nodes-6381.conf
3.4.2、redis-02
# redis-02
redis-server /usr/local/redis-6.2.5/etc/6379.conf
redis-server /usr/local/redis-6.2.5/etc/6380.conf
redis-server /usr/local/redis-6.2.5/etc/6381.conf
redis-02 ~]# netstat -tunlp | grep redis
tcp 0 0 0.0.0.0:6379 0.0.0.0:* LISTEN 27761/redis-server
tcp 0 0 0.0.0.0:6380 0.0.0.0:* LISTEN 27767/redis-server
tcp 0 0 0.0.0.0:6381 0.0.0.0:* LISTEN 27773/redis-server
tcp 0 0 0.0.0.0:16379 0.0.0.0:* LISTEN 27761/redis-server
tcp 0 0 0.0.0.0:16380 0.0.0.0:* LISTEN 27767/redis-server
tcp 0 0 0.0.0.0:16381 0.0.0.0:* LISTEN 27773/redis-server
redis-02 ~]# ll /usr/local/redis-6.2.5/data/
总用量 12
-rw-r--r--. 1 root root 114 5月 16 20:53 nodes-6379.conf
-rw-r--r--. 1 root root 114 5月 16 20:53 nodes-6380.conf
-rw-r--r--. 1 root root 114 5月 16 20:53 nodes-6381.conf
4、将六个节点合成一个集群
4.1、前提条件
组合之前，请确保所有redis实例启动后，nodes-xxxx.conf文件都生成正常。
4.2、合成集群
4.2.1、报错的解决
[ERR] Node 192.168.10.19:6380 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.
解决方法
将所有的节点都flushall一下或者重置集群：
redis-cli -p 6379 flushall
redis-cli -p 6380 flushall
redis-cli -p 6381 flushall
redis-cli -p 6380 CLUSTER reset
redis-cli -p 6379 CLUSTER reset
redis-cli -p 6381 CLUSTER reset
4.2.2、合并成为一个集群
redis-02 ~]# redis-cli --cluster create --cluster-replicas 1 192.168.10.19:6379 192.168.10.19:6380 192.168.10.19:6381 192.168.10.20:6379 192.168.10.20:6380 192.168.10.20:6381
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.168.10.20:6381 to 192.168.10.19:6379
Adding replica 192.168.10.19:6381 to 192.168.10.20:6379
Adding replica 192.168.10.20:6380 to 192.168.10.19:6380
M: 124c4d088133786aaebe5820f8d8fc093dd3289e 192.168.10.19:6379
slots:[0-5460] (5461 slots) master
M: 9a7befa99f0a574cd18b17d21c725c69cf43be23 192.168.10.19:6380
slots:[10923-16383] (5461 slots) master
S: 24813b313e72693904292fea9acfefeae46e6b71 192.168.10.19:6381
replicates 5b38b6e5f726d78f75780619ac383d53b08afa84
M: 5b38b6e5f726d78f75780619ac383d53b08afa84 192.168.10.20:6379
slots:[5461-10922] (5462 slots) master
S: c3ddf00c20e389802d7d0ffb0747bed1a6c6a976 192.168.10.20:6380
replicates 9a7befa99f0a574cd18b17d21c725c69cf43be23
S: 794078cb7c59e6d73aa3e2c58b55487ee064cafe 192.168.10.20:6381
replicates 124c4d088133786aaebe5820f8d8fc093dd3289e
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 192.168.10.19:6379)
M: 124c4d088133786aaebe5820f8d8fc093dd3289e 192.168.10.19:6379
slots:[0-5460] (5461 slots) master
1 additional replica(s)
S: 24813b313e72693904292fea9acfefeae46e6b71 192.168.10.19:6381
slots: (0 slots) slave
replicates 5b38b6e5f726d78f75780619ac383d53b08afa84
M: 5b38b6e5f726d78f75780619ac383d53b08afa84 192.168.10.20:6379
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: c3ddf00c20e389802d7d0ffb0747bed1a6c6a976 192.168.10.20:6380
slots: (0 slots) slave
replicates 9a7befa99f0a574cd18b17d21c725c69cf43be23
S: 794078cb7c59e6d73aa3e2c58b55487ee064cafe 192.168.10.20:6381
slots: (0 slots) slave
replicates 124c4d088133786aaebe5820f8d8fc093dd3289e
M: 9a7befa99f0a574cd18b17d21c725c69cf43be23 192.168.10.19:6380
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered. # 说明集群成功
属性参数解析：
一个集群至少要有三个主节点。
选项 --cluster-replicas 1 表示我们希望为集群中的每个主节点创建一个从节点。
分配原则尽量保证每个主数据库运行在不同的IP地址，每个从库和主库不在一个IP地址上。
4.2.3、查询节点状态
]# redis-cli cluster nodes
c3ddf00c20e389802d7d0ffb0747bed1a6c6a976 192.168.10.20:6380@16380 slave 9a7befa99f0a574cd18b17d21c725c69cf43be23 0 1684248605230 2 connected
9a7befa99f0a574cd18b17d21c725c69cf43be23 192.168.10.19:6380@16380 master - 0 1684248608248 2 connected 10923-16383
794078cb7c59e6d73aa3e2c58b55487ee064cafe 192.168.10.20:6381@16381 slave 124c4d088133786aaebe5820f8d8fc093dd3289e 0 1684248607241 1 connected
5b38b6e5f726d78f75780619ac383d53b08afa84 192.168.10.20:6379@16379 myself,master - 0 1684248605000 4 connected 5461-10922
24813b313e72693904292fea9acfefeae46e6b71 192.168.10.19:6381@16381 slave 5b38b6e5f726d78f75780619ac383d53b08afa84 0 1684248606237 4 connected
124c4d088133786aaebe5820f8d8fc093dd3289e 192.168.10.19:6379@16379 master - 0 1684248605000 1 connected 0-5460
4.3、写入数据测试
4.3.1、-c 采用集群策略连接，设置数据会自动切换到相应的写主机
# 设置值
redis-02 ~]# redis-cli -c
127.0.0.1:6379> set name cyc
OK
127.0.0.1:6379> get name
"cyc"
# 其它节点获取值
redis-01 ~]# redis-cli -c
127.0.0.1:6379> get name
-> Redirected to slot [5798] located at 192.168.10.20:6379
"cyc"
4.4、什么是slots
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
一个 Redis 集群包含 16384 个插槽（hash slot）， 数据库中的每个键都属于这 16384 个插槽的其中一个，
集群使用公式 CRC16(key) % 16384 来计算键 key 属于哪个槽， 其中 CRC16(key) 语句用于计算键 key 的 CRC16 校验和 。
集群中的每个节点负责处理一部分插槽。 举个例子， 如果一个集群可以有主节点， 其中：
节点 A 负责处理 0 号至 5460 号插槽。
节点 B 负责处理 5461 号至 10922 号插槽。
节点 C 负责处理 10923 号至 16383 号插槽。
4.5、集群查询
4.5.1、在集群中录入值
在redis-cli每次录入、查询键值，redis都会计算出该key应该送往的插槽，如果不是该客户端对应服务器的插槽，redis会报错，并告知应前往的redis实例地址和端口。
redis-cli客户端提供了 –c 参数实现自动重定向。
如 redis-cli -c –p 6379 登入后，再录入、查询键值对可以自动重定向。
不在一个slot下的键值，是不能使用mget,mset等多键操作。
127.0.0.1:6379> mset k1 v1 k2 v2 k3 v3
(error) CROSSSLOT Keys in request don't hash to the same slot
可以通过{}来定义组的概念，从而使key中{}内相同内容的键值对放到一个slot中去。
127.0.0.1:6379> mset k1{cust} v1 k2{cust} v2 k3{cust} v3
-> Redirected to slot [4847] located at 192.168.10.19:6379
OK
4.5.2、查询集群中的值
192.168.10.19:6379> CLUSTER KEYSLOT cust
(integer) 4847
192.168.10.19:6379> CLUSTER COUNTKEYSINSLOT 4847
(integer) 3
192.168.10.19:6379> CLUSTER GETKEYSINSLOT 4847 10
1) "k1{cust}"
2) "k2{cust}"
3) "k3{cust}"
4.6、故障恢复
4.6.1、停掉其中一个master
]# redis-cli -c
127.0.0.1:6379> shutdown
4.6.2、查询日志情况
]# tail -f /usr/local/redis-6.2.5/log/6380.log
29702:M 16 May 2023 22:48:04.929 * Background saving started by pid 30056
30056:C 16 May 2023 22:48:04.933 * DB saved on disk
30056:C 16 May 2023 22:48:04.933 * RDB: 4 MB of memory used by copy-on-write
29702:M 16 May 2023 22:48:05.005 * Background saving terminated with success
29702:M 16 May 2023 22:48:05.006 * Synchronization with replica 192.168.10.20:6380 succeeded
29702:M 16 May 2023 22:48:07.924 # Cluster state changed: ok
29702:M 16 May 2023 23:29:01.296 * FAIL message received from 24813b313e72693904292fea9acfefeae46e6b71 about 124c4d088133786aaebe5820f8d8fc093dd3289e
29702:M 16 May 2023 23:29:01.296 # Cluster state changed: fail
29702:M 16 May 2023 23:29:02.005 # Failover auth granted to 794078cb7c59e6d73aa3e2c58b55487ee064cafe for epoch 7
29702:M 16 May 2023 23:29:02.010 # Cluster state changed: ok
4.6.3、查询集群节点
127.0.0.1:6379> CLUSTER nodes
c3ddf00c20e389802d7d0ffb0747bed1a6c6a976 192.168.10.20:6380@16380 slave 9a7befa99f0a574cd18b17d21c725c69cf43be23 0 1684251060731 2 connected
9a7befa99f0a574cd18b17d21c725c69cf43be23 192.168.10.19:6380@16380 master - 0 1684251061742 2 connected 10923-16383
794078cb7c59e6d73aa3e2c58b55487ee064cafe 192.168.10.20:6381@16381 master - 0 1684251059718 7 connected 0-5460 # 这个是就是从节点接管为master
5b38b6e5f726d78f75780619ac383d53b08afa84 192.168.10.20:6379@16379 myself,master - 0 1684251059000 4 connected 5461-10922
24813b313e72693904292fea9acfefeae46e6b71 192.168.10.19:6381@16381 slave 5b38b6e5f726d78f75780619ac383d53b08afa84 0 1684251058000 4 connected
124c4d088133786aaebe5820f8d8fc093dd3289e 192.168.10.19:6379@16379 master,fail - 1684250926110 1684250921000 1 disconnected # 已经失败，停用，master还是有3个，说明已经自动从变成主
4.6.4、恢复下掉的节点
redis-server /usr/local/redis-6.2.5/etc/6379.conf
4.6.5、分析集群节点状态【恢复后的节点，将一直变成slave】
127.0.0.1:6379> CLUSTER nodes
c3ddf00c20e389802d7d0ffb0747bed1a6c6a976 192.168.10.20:6380@16380 slave 9a7befa99f0a574cd18b17d21c725c69cf43be23 0 1684251327352 2 connected
9a7befa99f0a574cd18b17d21c725c69cf43be23 192.168.10.19:6380@16380 master - 0 1684251326000 2 connected 10923-16383
794078cb7c59e6d73aa3e2c58b55487ee064cafe 192.168.10.20:6381@16381 master - 0 1684251328358 7 connected 0-5460
5b38b6e5f726d78f75780619ac383d53b08afa84 192.168.10.20:6379@16379 myself,master - 0 1684251327000 4 connected 5461-10922
24813b313e72693904292fea9acfefeae46e6b71 192.168.10.19:6381@16381 slave 5b38b6e5f726d78f75780619ac383d53b08afa84 0 1684251326000 4 connected
124c4d088133786aaebe5820f8d8fc093dd3289e 192.168.10.19:6379@16379 slave 794078cb7c59e6d73aa3e2c58b55487ee064cafe 0 1684251326000 7 connected # 这个就是恢复的节点
4.7、如果所有某一段插槽的主从节点都宕掉，redis服务是否还能继续?
如果某一段插槽的主从都挂掉，而cluster-require-full-coverage 为yes ，那么 ，整个集群都挂掉
如果某一段插槽的主从都挂掉，而cluster-require-full-coverage 为no ，那么，该插槽数据全都不能使用，也无法存储。
redis.conf中的参数 cluster-require-full-coverage
5、集群优点与缺点
5.1、优点
5.2、缺点
多键操作是不被支持的
多键的Redis事务是不被支持的。lua脚本不被支持
由于集群方案出现较晚，很多公司已经采用了其他的集群方案，而代理或者客户端分片的方案想要迁移至redis cluster，需要整体迁移而不是逐步过渡，复杂度较大。
