Redis Cluster集群架构实现(四)--技术流ken
Redis集群简介
通过前面三篇博客的介绍《Redis基础认识及常用命令使用(一)--技术流ken》，《Redis基础知识补充及持久化、备份介绍(二)--技术流ken》，《Redis主从复制、多实例、高可用(三)--技术流ken》，现在已经对redis的基础知识，常用命令，持久化，备份，主从复制，多实例的安装以及redis的高可用熟练掌握了。本篇博客将介绍redis cluster集群，也是一个比较复杂的内容，本篇博客将采用较为简洁的方式来呈现redis集群。
有关redis集群的介绍可以参考下面我摘自redis官网的简介。
Redis 集群是一个可以在多个 Redis 节点之间进行数据共享的设施（installation）。
Redis 集群不支持那些需要同时处理多个键的 Redis 命令， 因为执行这些命令需要在多个 Redis 节点之间移动数据， 并且在高负载的情况下， 这些命令将降低 Redis 集群的性能， 并导致不可预测的行为。
Redis 集群通过分区（partition）来提供一定程度的可用性（availability）： 即使集群中有一部分节点失效或者无法进行通讯， 集群也可以继续处理命令请求。
Redis 集群提供了以下两个好处：
- 将数据自动切分（split）到多个节点的能力。
- 当集群中的一部分节点失效或者无法进行通讯时， 仍然可以继续处理命令请求的能力。 --摘自redis官网
总而言之，redis集群实现了数据的共享以及去中心化。
Redis集群实现
实现redis集群，现在采用一台服务器安装6个redis实例，有关如果安装多实例的详细讲解请参考我上篇博客《Redis主从复制、多实例、高可用(三)--技术流ken》
6个多实例，三个为主节点，三个为从节点
环境：
CentOS Linux release 7.5.1804 (Core)
redis-4.0.11
创建多实例
第一步：创建多实例数据目录
root@ken ~]# mkdir /ken [root@ken ~]# cd /ken [root@ken ken]# mkdir 6379 6380 6381 6382 6383 6384 [root@ken ken]# ls 6379 6380 6381 6382 6383 6384
第二步：上传redis安装包
这里使用的是redis-4.0.11的版本，可以在redis官方网站进行下载https://redis.io/
[root@ken ~]# rz
第三步：解压安装包
[root@ken ~]# tar xf redis-4.0.11.tar.gz
第四步：复制解压安装包的配置文件到6379目录下
[root@ken ~]# cp redis-4.0.11/redis.conf /ken/6379/
第五步：移动redis解压包至/usr/local/redis下
[root@ken ~]# mv redis-4.0.11 /usr/local/redis
第六步：编译安装
[root@ken ~]# cd /usr/local/redis
[root@ken redis]# make && make install
[root@ken ~]# ln /usr/local/redis/src/ /bin -s
第七步：修改配置文件
[root@ken ~]# grep -E -v "^#|^$" /ken/6379/redis.conf bind 10.220.5.137 #绑定本机ip地址 protected-mode yes port 6379 #监听端口 tcp-backlog 511 timeout 0 tcp-keepalive 300 daemonize yes #后台运行 supervised no pidfile /ken/6379/redis_6379.pid #pid文件保存位置 loglevel notice logfile "" databases 16 always-show-logo yes save 900 1 save 300 10 save 60 10000 stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum yes dbfilename dump.rdb dir /ken/6379/ #rdb文件保存位置 slave-serve-stale-data yes slave-read-only yes repl-diskless-sync no repl-diskless-sync-delay 5 repl-disable-tcp-nodelay no slave-priority 100 lazyfree-lazy-eviction no lazyfree-lazy-expire no lazyfree-lazy-server-del no slave-lazy-flush no appendonly no appendfilename "appendonly.aof" appendfsync everysec no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb aof-load-truncated yes aof-use-rdb-preamble no lua-time-limit 5000 cluster-enabled yes #开启集群 cluster-config-file nodes-6379.conf #集群文件名称 cluster-node-timeout 15000 #集群超时时间 slowlog-log-slower-than 10000 slowlog-max-len 128 latency-monitor-threshold 0 notify-keyspace-events "" hash-max-ziplist-entries 512 hash-max-ziplist-value 64 list-max-ziplist-size -2 list-compress-depth 0 set-max-intset-entries 512 zset-max-ziplist-entries 128 zset-max-ziplist-value 64 hll-sparse-max-bytes 3000 activerehashing yes client-output-buffer-limit normal 0 0 0 client-output-buffer-limit slave 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 hz 10 aof-rewrite-incremental-fsync yes
第八步：把上面这个修改过的文件复制到其他的实例数据目录之下
[root@ken ~]# cp /ken/6379/redis.conf /ken/6380/ [root@ken ~]# cp /ken/6379/redis.conf /ken/6381/ [root@ken ~]# cp /ken/6379/redis.conf /ken/6382/ [root@ken ~]# cp /ken/6379/redis.conf /ken/6383/ [root@ken ~]# cp /ken/6379/redis.conf /ken/6384/
第九步:修改配置文件
只要使用sed即可进行修改
[root@ken ~]# sed -i 's/6379/6380/g' /ken/6380/redis.conf [root@ken ~]# sed -i 's/6379/6381/g' /ken/6381/redis.conf [root@ken ~]# sed -i 's/6379/6382/g' /ken/6382/redis.conf [root@ken ~]# sed -i 's/6379/6383/g' /ken/6383/redis.conf [root@ken ~]# sed -i 's/6379/6384/g' /ken/6384/redis.conf
第十步：启动各个实例
[root@ken ~]# redis-server /ken/6380/redis.conf [root@ken ~]# redis-server /ken/6381/redis.conf [root@ken ~]# redis-server /ken/6382/redis.conf [root@ken ~]# redis-server /ken/6383/redis.conf [root@ken ~]# redis-server /ken/6384/redis.conf [root@ken ~]# ss -tnl State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 10.220.5.137:16380 *:* LISTEN 0 128 10.220.5.137:16381 *:* LISTEN 0 128 10.220.5.137:16382 *:* LISTEN 0 128 10.220.5.137:16383 *:* LISTEN 0 128 10.220.5.137:16384 *:* LISTEN 0 128 *:10050 *:* LISTEN 0 128 *:10051 *:* LISTEN 0 128 10.220.5.137:6379 *:* LISTEN 0 128 10.220.5.137:6380 *:* LISTEN 0 128 10.220.5.137:6381 *:* LISTEN 0 128 10.220.5.137:6382 *:* LISTEN 0 128 10.220.5.137:6383 *:* LISTEN 0 128 *:111 *:* LISTEN 0 128 10.220.5.137:6384 *:* LISTEN 0 128 *:22 *:* LISTEN 0 128 10.220.5.137:16379 *:* LISTEN 0 128 :::10050 :::* LISTEN 0 128 :::10051 :::* LISTEN 0 70 :::3306 :::* LISTEN 0 128 :::111 :::* LISTEN 0 128 :::80 :::* LISTEN 0 128 :::22 :::*
安装ruby2.3
实现redis cluster功能，依赖redis-trib.rb，而这个工具是依赖一个ruby开发工具包的，所以需要安装ruby环境，并安装依赖包
第一步:上传解压安装包
[root@ken ~]# rz [root@ken ~]# tar xf ruby-2.3.5.tar.gz
第二步：编译安装
建议虚拟机内存至少1个G以上
[root@ken ~]# cd ruby-2.3.5/ [root@ken ruby-2.3.5]# ./configure --prefix=/usr/local/ruby && make && make install
第三步：安装redis-trib.rb的依赖
[root@ken ~]# ln -s /usr/local/ruby/bin/gem /bin [root@ken ~]# gem install -l redis-3.3.0.gem Successfully installed redis-3.3.0 Parsing documentation for redis-3.3.0 Installing ri documentation for redis-3.3.0 Done installing documentation for redis after 0 seconds 1 gem installed
[root@ken ~]# ln -s /usr/local/ruby/bin/ruby /bin
获取集群帮助
只需要输入redis-trib.rb回车即可
[root@ken ~]# redis-trib.rb Usage: redis-trib <command> <options> <arguments ...> create host1:port1 ... hostN:portN --replicas <arg> check host:port info host:port fix host:port --timeout <arg> reshard host:port --from <arg> --to <arg> --slots <arg> --yes --timeout <arg> --pipeline <arg> rebalance host:port --weight <arg> --auto-weights --use-empty-masters --timeout <arg> --simulate --pipeline <arg> --threshold <arg> add-node new_host:new_port existing_host:existing_port --slave --master-id <arg> del-node host:port node_id set-timeout host:port milliseconds call host:port command arg arg .. arg import host:port --from <arg> --copy --replace help (show this help) For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.
创建redis集群
命令的意义如下：
- 给定
redis-trib.rb程序的命令是create， 这表示我们希望创建一个新的集群。 - 选项
--replicas 1表示我们希望为集群中的每个主节点创建一个从节点。 - 之后跟着的其他参数则是实例的地址列表， 我们希望程序使用这些地址所指示的实例来创建新集群。
简单来说， 以上命令的意思就是让 redis-trib 程序创建一个包含三个主节点和三个从节点的集群。
接着， redis-trib 会打印出一份预想中的配置给你看， 如果你觉得没问题的话， 就可以输入 yes ， redis-trib 就会将这份配置应用到集群当中：
[root@ken ~]# redis-trib.rb create --replicas 1 10.220.5.137:6379 10.220.5.137:6380 10.220.5.137:6381 10.220.5.137:6382 10.220.5.137:6383 10.220.5.137:6384 >>> Creating cluster >>> Performing hash slots allocation on 6 nodes... Using 3 masters: 10.220.5.137:6379 10.220.5.137:6380 10.220.5.137:6381 Adding replica 10.220.5.137:6383 to 10.220.5.137:6379 Adding replica 10.220.5.137:6384 to 10.220.5.137:6380 Adding replica 10.220.5.137:6382 to 10.220.5.137:6381 >>> Trying to optimize slaves allocation for anti-affinity [WARNING] Some slaves are in the same host as their master M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379 slots:0-5460 (5461 slots) master M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380 slots:5461-10922 (5462 slots) master M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381 slots:10923-16383 (5461 slots) master S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382 replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5 S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383 replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384 replicates ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 Can I set the above configuration? (type 'yes' to accept): yes #输入yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join..... >>> Performing Cluster Check (using node 10.220.5.137:6379) M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379 slots:0-5460 (5461 slots) master 1 additional replica(s) M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381 slots:10923-16383 (5461 slots) master 1 additional replica(s) M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380 slots:5461-10922 (5462 slots) master 1 additional replica(s) S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383 slots: (0 slots) slave replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384 slots: (0 slots) slave replicates ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382 slots: (0 slots) slave replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
Redis集群测试
测试 Redis 集群比较简单的办法就是使用 redis-rb-cluster 或者 redis-cli ， 接下来将使用 redis-cli 为例来进行演示：
第一步：登录集群中
一定要加个-c，后面输入哪个端口都可以，这就是redis的区中心化思想
[root@ken ~]# redis-cli -c -h 10.220.5.137 -p 6379
第二步：创建key
可以发现创建的key被分配到了不同的节点
10.220.5.137:6379> keys * (empty list or set) 10.220.5.137:6379> set name ken -> Redirected to slot [5798] located at 10.220.5.137:6380 OK 10.220.5.137:6380> set addr jiangsu -> Redirected to slot [12790] located at 10.220.5.137:6381 OK 10.220.5.137:6381> set tel 123445 -> Redirected to slot [7485] located at 10.220.5.137:6380 OK 10.220.5.137:6380> set ege 25 OK 10.220.5.137:6380> set gender male -> Redirected to slot [15355] located at 10.220.5.137:6381 OK 10.220.5.137:6381> keys * 1) "addr" 2) "gender"
第三步：获取key
在端口6381的节点之上没有tel这个key,但是我们仍然可以使用get tel获取到值，这就实现了redis集群的数据共享
10.220.5.137:6381> keys * 1) "addr" 2) "gender" 10.220.5.137:6381> get tel -> Redirected to slot [7485] located at 10.220.5.137:6380 "123445"
在集群中添加新的节点
第一步：创建一个新的实例
[root@ken ~]# mkdir /ken/6385 [root@ken ~]# cp /ken/6379/redis.conf /ken/6385/ [root@ken ~]# sed -i 's/6379/6385/g' /ken/6385/redis.conf [root@ken ~]# redis-server /ken/6385/redis.conf 10461:C 15 Nov 21:21:11.646 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 10461:C 15 Nov 21:21:11.646 # Redis version=4.0.11, bits=64, commit=00000000, modified=0, pid=10461, just started 10461:C 15 Nov 21:21:11.646 # Configuration loaded
第二步：添加该节点到集群中
命令中的 add-node 表示我们要让 redis-trib 将一个节点添加到集群里面， add-node 之后跟着的是新节点的 IP 地址和端口号， 再之后跟着的是集群中任意一个已存在节点的 IP 地址和端口号， 这里我们使用的是 10.220.5.137:6379 。
[root@ken ~]# redis-trib.rb add-node 10.220.5.137:6385 10.220.5.137:6379 >>> Adding node 10.220.5.137:6385 to cluster 10.220.5.137:6379 >>> Performing Cluster Check (using node 10.220.5.137:6379) M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379 slots:0-5460 (5461 slots) master 1 additional replica(s) M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381 slots:10923-16383 (5461 slots) master 1 additional replica(s) M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380 slots:5461-10922 (5462 slots) master 1 additional replica(s) S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383 slots: (0 slots) slave replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384 slots: (0 slots) slave replicates ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382 slots: (0 slots) slave replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. >>> Send CLUSTER MEET to node 10.220.5.137:6385 to make it join the cluster. [OK] New node added correctly.
第三步:查看
通过 cluster nodes 命令， 我们可以确认新节点10.220.5.137:6385 已经被添加到集群里面了
10.220.5.137:6379> CLUSTER nodes 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379@16379 myself,master - 0 1542288244000 1 connected 0-5460 daf9464ef45d0c73e1ee18f7cc7ddbc7d9719f2d 10.220.5.137:6385@16385 master - 0 1542288242824 0 connected ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381@16381 master - 0 1542288244000 3 connected 10923-16383 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380@16380 master - 0 1542288242000 2 connected 5461-10922 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383@16383 slave 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 0 1542288242000 5 connected b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384@16384 slave ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 0 1542288244852 6 connected 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382@16382 slave 57753754f4c89054ab14e8ec517604c4fc4c8ed5 0 1542288243844 4 connected
第四步：使添加的新节点为从节点
如果我们打算让新节点成为 10.220.5.137:6379 的从节点， 那么我们只要用客户端连接上新节点， 然后执行以下命令就可以了
[root@ken ~]# redis-cli -c -h 10.220.5.137 -p 6385 10.220.5.137:6385> CLUSTER REPLICATE 57753754f4c89054ab14e8ec517604c4fc4c8ed5 OK
其中57753754f4c89054ab14e8ec517604c4fc4c8ed5是主节点10.220.5.137:6379的id
对redis中的数据做重新分片
[root@ken ~]# redis-trib.rb reshard 10.220.5.137:6379
你只需要指定集群中其中一个节点的地址， redis-trib 就会自动找到集群中的其他节点。
How many slots do you want to move (from 1 to 16384)? 1000
我们将打算移动的槽数量设置为 1000 个。
What is the receiving node ID? 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d
定目标需要使用节点的 ID ， 而不是 IP 地址和端口。 比如说， 我们打算使用集群的第一个主节点来作为目标， 它的 IP 地址和端口是 10.220.5.137:6380 ， 而节点 ID 则是3a9aa9592afc594c7e4206cc82ffb37d46a5b23d ， 那么我们应该向 redis-trib 提供节点的 ID
Source node #1:ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8
接着， redis-trib 会向你询问重新分片的源节点（source node）， 也即是， 要从哪个节点中取出 1000 个哈希槽， 并将这些槽移动到目标节点上面。
Source node #2:done
Do you want to proceed with the proposed reshard plan (yes/no)? yes
输入 yes 并使用按下回车之后， redis-trib 就会正式开始执行重新分片操作， 将指定的哈希槽从源节点一个个地移动到目标节点上面
Moving slot 12905 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12906 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12907 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12908 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12909 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12910 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12911 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12912 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12913 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12914 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12915 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12916 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12917 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12918 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12919 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12920 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12921 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12922 from 10.220.5.137:6381 to 10.220.5.137:6380:
在重新分片操作执行完毕之后， 可以使用以下命令来检查集群是否正常
[root@ken ~]# redis-trib.rb check 10.220.5.137:6379 >>> Performing Cluster Check (using node 10.220.5.137:6379) M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379 slots:0-5460 (5461 slots) master 2 additional replica(s) S: daf9464ef45d0c73e1ee18f7cc7ddbc7d9719f2d 10.220.5.137:6385 slots: (0 slots) slave replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5 M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381 slots:12923-16383 (3461 slots) master 1 additional replica(s) M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380 slots:5461-12922 (7462 slots) master 1 additional replica(s) S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383 slots: (0 slots) slave replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384 slots: (0 slots) slave replicates ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382 slots: (0 slots) slave replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
删除节点
如果节点中有slot，那么需要先将slot 执行reshard给其他节点，然后才能执行删除操作
1. 删除空slot
删除10.220.5.137:6385
[root@ken ~]# redis-trib.rb check 10.220.5.137:6379 >>> Performing Cluster Check (using node 10.220.5.137:6379) M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379 slots:0-5460 (5461 slots) master 2 additional replica(s) S: daf9464ef45d0c73e1ee18f7cc7ddbc7d9719f2d 10.220.5.137:6385 slots: (0 slots) slave replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5 M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381 slots:12923-16383 (3461 slots) master 1 additional replica(s) ...
执行如下命令
[root@ken ~]# redis-trib.rb del-node 10.220.5.137:6379 daf9464ef45d0c73e1ee18f7cc7ddbc7d9719f2d >>> Removing node daf9464ef45d0c73e1ee18f7cc7ddbc7d9719f2d from cluster 10.220.5.137:6379 >>> Sending CLUSTER FORGET messages to the cluster... >>> SHUTDOWN the node.
查看发现6385节点已经被移除
[root@ken ~]# redis-trib.rb check 10.220.5.137:6379 >>> Performing Cluster Check (using node 10.220.5.137:6379) M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379 slots:0-5460 (5461 slots) master 1 additional replica(s) M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381 slots:12923-16383 (3461 slots) master 1 additional replica(s) M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380 slots:5461-12922 (7462 slots) master 1 additional replica(s) S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383 slots: (0 slots) slave replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384 slots: (0 slots) slave replicates ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382 slots: (0 slots) slave replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
2.删除带有slot的节点
提示有数据报错
[root@ken ~]# redis-trib.rb del-node 10.220.5.137:6379 ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 >>> Removing node ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 from cluster 10.220.5.137:6379 [ERR] Node 10.220.5.137:6381 is not empty! Reshard data away and try again.
需要重新分片
[root@ken ~]# redis-trib.rb reshard 10.220.5.137:6379 >>> Performing Cluster Check (using node 10.220.5.137:6379) M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379 slots:0-5460 (5461 slots) master 1 additional replica(s) M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381 slots:12923-16383 (3461 slots) master 1 additional replica(s) M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380 slots:5461-12922 (7462 slots) master 1 additional replica(s) S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383 slots: (0 slots) slave replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384 slots: (0 slots) slave replicates ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382 slots: (0 slots) slave replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. How many slots do you want to move (from 1 to 16384)? 3461 What is the receiving node ID? 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d Please enter all the source node IDs. Type 'all' to use all the nodes as source nodes for the hash slots. Type 'done' once you entered all the source nodes IDs. Source node #1:ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 Source node #2:done
再检查slot已经为空
>>> Performing Cluster Check (using node 10.220.5.137:6379) M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379 slots:0-5460 (5461 slots) master 1 additional replica(s) M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381 #已经为空 slots: (0 slots) master 0 additional replica(s) M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380 slots:5461-16383 (10923 slots) master 2 additional replica(s) S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383 slots: (0 slots) slave replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384 slots: (0 slots) slave replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382 slots: (0 slots) slave replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
再执行删除操作
[root@ken ~]# redis-trib.rb del-node 10.220.5.137:6379 ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 >>> Removing node ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 from cluster 10.220.5.137:6379 >>> Sending CLUSTER FORGET messages to the cluster... >>> SHUTDOWN the node.
再次查看。6381节点已经被删除
[root@ken ~]# redis-trib.rb check 10.220.5.137:6379 >>> Performing Cluster Check (using node 10.220.5.137:6379) M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379 slots:0-5460 (5461 slots) master 1 additional replica(s) M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380 slots:5461-16383 (10923 slots) master 2 additional replica(s) S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383 slots: (0 slots) slave replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384 slots: (0 slots) slave replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382 slots: (0 slots) slave replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
