redis3.0 集群实战2 - 集群功能实战
1 集群基本操作
1.1 查看当前集群状态
使用redis-trib.rb check功能查看对应的节点的状态:
[root@bogon bin]# ./redis-trib.rb check 127.0.0.1:7000 Connecting to node 127.0.0.1:7000: OK Connecting to node 127.0.0.1:7002: OK Connecting to node 127.0.0.1:7001: OK Performing Cluster Check (using node 127.0.0.1:7000) M: 36fce4bf6c829797c433c08b045a8c482e07fbc1 127.0.0.1:7000 slots:0-5460 (5461 slots) master 0 additional replica(s) M: e89c01d7cc2a83f072f66d0c2fb1f5cd92c66888 127.0.0.1:7002 slots:10923-16383 (5461 slots) master 0 additional replica(s) M: 7e46925ca9781a403facf450aa0b4b8149ad9c27 127.0.0.1:7001 slots:5461-10922 (5462 slots) master 0 additional replica(s) [OK] All nodes agree about slots configuration. Check for open slots... Check slots coverage... [OK] All 16384 slots covered.
如上图所示,当前我们有三个节点7000到7002,默认总共16384个slot,平均一个节点5461个slot
或者我们也可以使用redis-cli连接上任意一个结点,执行cluster nodes查看结点状态:
[root@bogon bin]# ./redis-cli -c -p 7000 cluster nodes e89c01d7cc2a83f072f66d0c2fb1f5cd92c66888 127.0.0.1:7002 master - 0 1440144105378 3 connected 10923-16383 36fce4bf6c829797c433c08b045a8c482e07fbc1 127.0.0.1:7000 myself,master - 0 0 1 connected 0-5460 7e46925ca9781a403facf450aa0b4b8149ad9c27 127.0.0.1:7001 master - 0 1440144106398 2 connected 5461-10922
1.2 增加一个主节点
1 启动7003的redis实例:
[root@bogon bin]# ps aux | grep redis root 4212 0.1 0.1 140836 7252 ? Ssl Aug20 0:41 ./redis-server *:7000 [cluster] root 4214 0.1 0.1 140836 7296 ? Ssl Aug20 0:41 ./redis-server *:7001 [cluster] root 4218 0.1 0.1 140836 7228 ? Ssl Aug20 0:41 ./redis-server *:7002 [cluster] root 10817 0.0 0.1 20160 5180 pts/0 S+ 00:41 0:00 ./redis-cli -c -p 7000 root 11223 0.0 0.1 140836 7460 ? Ssl 00:54 0:00 ./redis-server *:7003 [cluster]
2 使用 redis-cli cluster nodes查看当前集群状态,跟1.1一样,7003实例还没加进来
[root@bogon bin]# ./redis-cli -c -p 7000 cluster nodes e89c01d7cc2a83f072f66d0c2fb1f5cd92c66888 127.0.0.1:7002 master - 0 1440144105378 3 connected 10923-16383 36fce4bf6c829797c433c08b045a8c482e07fbc1 127.0.0.1:7000 myself,master - 0 0 1 connected 0-5460 7e46925ca9781a403facf450aa0b4b8149ad9c27 127.0.0.1:7001 master - 0 1440144106398 2 connected 5461-10922
3 使用redis-trib.rb add-node增加结点
[root@bogon bin]# ./redis-trib.rb add-node 127.0.0.1:7003 127.0.0.1:7000 Adding node 127.0.0.1:7003 to cluster 127.0.0.1:7000 Connecting to node 127.0.0.1:7000: OK Connecting to node 127.0.0.1:7002: OK Connecting to node 127.0.0.1:7001: OK Performing Cluster Check (using node 127.0.0.1:7000) M: 36fce4bf6c829797c433c08b045a8c482e07fbc1 127.0.0.1:7000 slots:0-5460 (5461 slots) master 0 additional replica(s) M: e89c01d7cc2a83f072f66d0c2fb1f5cd92c66888 127.0.0.1:7002 slots:10923-16383 (5461 slots) master 0 additional replica(s) M: 7e46925ca9781a403facf450aa0b4b8149ad9c27 127.0.0.1:7001 slots:5461-10922 (5462 slots) master 0 additional replica(s) [OK] All nodes agree about slots configuration. Check for open slots... Check slots coverage... [OK] All 16384 slots covered. Connecting to node 127.0.0.1:7003: OK Send CLUSTER MEET to node 127.0.0.1:7003 to make it join the cluster. [OK] New node added correctly.
4 使用redis-cli查看当前cluster的状态
[root@bogon bin]# ./redis-cli -c -p 7000 cluster nodes 245d95199dcc1e01880b2876fcab8a75b1867b19 127.0.0.1:7003 master - 0 1440144104360 0 connected e89c01d7cc2a83f072f66d0c2fb1f5cd92c66888 127.0.0.1:7002 master - 0 1440144105378 3 connected 10923-16383 36fce4bf6c829797c433c08b045a8c482e07fbc1 127.0.0.1:7000 myself,master - 0 0 1 connected 0-5460 7e46925ca9781a403facf450aa0b4b8149ad9c27 127.0.0.1:7001 master - 0 1440144106398 2 connected 5461-10922
发现新节点已经连上cluster了,成为集群一份子,并且默认是master结点。这时候,我们已经可以使用redis-cli连接到该结点查询情况
[root@bogon bin]# ./redis-cli -c -p 7003 127.0.0.1:7003> set a b -> Redirected to slot [15495] located at 127.0.0.1:7002 OK 127.0.0.1:7002>
不过redis cluster并不会自动把hash分片迁移到该结点
5 使用redis-trib.rb rehashed重新设置分片
- ./redis-trib.rb reshard 127.0.0.1:7003执行rehash
- 需要再命令行中输入slots个数(迁移的slots数目)
- 需要填写迁移目标结点的id,这里我们希望迁移到7003上,所以把7003的id写上去(通过redis-cli cluster nodes查看对应id)
- 需要填写source node,可以填上对应的source节点id,或者写all,表示从所有节点中抽取1000个出来,这里写all
- 这里redis集群会列出来模拟迁移的点,确认后,填写yes,那么开始进行实例迁移
6 查看新的分片
127.0.0.1:7000> cluster nodes 245d95199dcc1e01880b2876fcab8a75b1867b19 127.0.0.1:7003 master - 0 1440145009719 4 connected 0-332 5461-5794 10923-11255 e89c01d7cc2a83f072f66d0c2fb1f5cd92c66888 127.0.0.1:7002 master - 0 1440145009206 3 connected 11256-16383 36fce4bf6c829797c433c08b045a8c482e07fbc1 127.0.0.1:7000 myself,master - 0 0 1 connected 333-5460 7e46925ca9781a403facf450aa0b4b8149ad9c27 127.0.0.1:7001 master - 0 1440145010228 2 connected 5795-10922
如上所示,可以看出,7003从剩余3个节点中抽取除了1000个slots出来,整体上抽取的相对比较均匀
1.3 增加一个从节点
1 启动7004的redis实例
[root@bogon bin]# ps aux | grep redis root 4212 0.1 0.1 140836 7240 ? Ssl Aug20 0:46 ./redis-server *:7000 [cluster] root 4214 0.1 0.1 140836 7296 ? Ssl Aug20 0:46 ./redis-server *:7001 [cluster] root 4218 0.1 0.1 140836 7292 ? Ssl Aug20 0:46 ./redis-server *:7002 [cluster] root 10817 0.0 0.1 20160 5192 pts/0 S+ 00:41 0:00 ./redis-cli -c -p 7000 root 11223 0.2 0.1 140836 7356 ? Ssl 00:54 0:05 ./redis-server *:7003 [cluster] root 12063 0.2 0.1 140836 7464 ? Ssl 01:34 0:00 ./redis-server *:7004 [cluster]
2 使用redis-trib.rb add-node增加7004节点
[root@bogon bin]# ./redis-trib.rb add-node 127.0.0.1:7004 127.0.0.1:7000 Adding node 127.0.0.1:7004 to cluster 127.0.0.1:7000 Connecting to node 127.0.0.1:7000: OK Connecting to node 127.0.0.1:7002: OK Connecting to node 127.0.0.1:7003: OK Connecting to node 127.0.0.1:7001: OK Performing Cluster Check (using node 127.0.0.1:7000) M: 36fce4bf6c829797c433c08b045a8c482e07fbc1 127.0.0.1:7000 slots:333-5460 (5128 slots) master 0 additional replica(s) M: e89c01d7cc2a83f072f66d0c2fb1f5cd92c66888 127.0.0.1:7002 slots:11256-16383 (5128 slots) master 0 additional replica(s) M: 245d95199dcc1e01880b2876fcab8a75b1867b19 127.0.0.1:7003 slots:0-332,5461-5794,10923-11255 (1000 slots) master 0 additional replica(s) M: 7e46925ca9781a403facf450aa0b4b8149ad9c27 127.0.0.1:7001 slots:5795-10922 (5128 slots) master 0 additional replica(s)
[OK] All nodes agree about slots configuration. Check for open slots... Check slots coverage... [OK] All 16384 slots covered. Connecting to node 127.0.0.1:7004: OK Send CLUSTER MEET to node 127.0.0.1:7004 to make it join the cluster. [OK] New node added correctly.
3 查看当前集群nodes状态
[root@bogon bin]# ./redis-cli -c -p 7000 cluster nodes 36fce4bf6c829797c433c08b045a8c482e07fbc1 127.0.0.1:7000 myself,master - 0 0 1 connected 333-5460 e89c01d7cc2a83f072f66d0c2fb1f5cd92c66888 127.0.0.1:7002 master - 0 1440147387481 3 connected 11256-16383 245d95199dcc1e01880b2876fcab8a75b1867b19 127.0.0.1:7003 master - 0 1440147389014 4 connected 0-332 5461-5794 10923-11255 f26de43ddafc2e8e000132c3b53473d42906b429 127.0.0.1:7004 master - 0 1440147387991 0 connected 7e46925ca9781a403facf450aa0b4b8149ad9c27 127.0.0.1:7001 master - 0 1440147388501 2 connected 5795-10922
默认在增加节点的时候都是master节点,因此刚加进去的7004节点也是master
4 将7004转成7001的slave节点
# 使用redis-cli连接到7004上,查看当前node的状态
[root@bogon bin]# ./redis-cli -p 7004 -c 127.0.0.1:7004> cluster nodes f26de43ddafc2e8e000132c3b53473d42906b429 127.0.0.1:7004 myself,master - 0 0 0 connected 245d95199dcc1e01880b2876fcab8a75b1867b19 127.0.0.1:7003 master - 0 1440147476431 4 connected 0-332 5461-5794 10923-11255 36fce4bf6c829797c433c08b045a8c482e07fbc1 127.0.0.1:7000 master - 0 1440147476944 1 connected 333-5460 7e46925ca9781a403facf450aa0b4b8149ad9c27 127.0.0.1:7001 master - 0 1440147477452 2 connected 5795-10922 e89c01d7cc2a83f072f66d0c2fb1f5cd92c66888 127.0.0.1:7002 master - 0 1440147475924 3 connected 11256-16383
# 使用cluster replicate <id>命令将7004设置为7001的从节点,其中id为7001的节点id
127.0.0.1:7004> cluster replicate 7e46925ca9781a403facf450aa0b4b8149ad9c27
OK
# 再次查看cluster nodes
127.0.0.1:7004> cluster nodes
f26de43ddafc2e8e000132c3b53473d42906b429 127.0.0.1:7004 myself,slave 7e46925ca9781a403facf450aa0b4b8149ad9c27 0 0 0 connected
245d95199dcc1e01880b2876fcab8a75b1867b19 127.0.0.1:7003 master - 0 1440147500851 4 connected 0-332 5461-5794 10923-11255
36fce4bf6c829797c433c08b045a8c482e07fbc1 127.0.0.1:7000 master - 0 1440147499834 1 connected 333-5460
7e46925ca9781a403facf450aa0b4b8149ad9c27 127.0.0.1:7001 master - 0 1440147500849 2 connected 5795-10922
e89c01d7cc2a83f072f66d0c2fb1f5cd92c66888 127.0.0.1:7002 master - 0 1440147500338 3 connected 11256-16383
# 当前7004已经成功成为7001的从节点了
1.4 主从自动切换
-
查看当前集群状态
[root@bogon bin]# ./redis-cli -p 7000 -c cluster nodes 36fce4bf6c829797c433c08b045a8c482e07fbc1 127.0.0.1:7000 myself,master - 0 0 5 connected 0-5794 10923-11255 e89c01d7cc2a83f072f66d0c2fb1f5cd92c66888 127.0.0.1:7002 master - 0 1440402734058 3 connected 11256-16383 245d95199dcc1e01880b2876fcab8a75b1867b19 127.0.0.1:7003 slave 36fce4bf6c829797c433c08b045a8c482e07fbc1 0 1440402733548 5 connected f26de43ddafc2e8e000132c3b53473d42906b429 127.0.0.1:7004 slave 7e46925ca9781a403facf450aa0b4b8149ad9c27 0 1440402734573 2 connected 7e46925ca9781a403facf450aa0b4b8149ad9c27 127.0.0.1:7001 master - 0 1440402735085 2 connected 5795-10922
如图所示,当前7000,7001,7002三个为主,7003为7000的slave,7004位7001的slave
-
构造数据
# 构造数据 for ((i=0;i<1000;i++)); do ./redis-cli -p 7000 -c set "key$i" "$i";done # 查看数据分片情况: [root@bogon bin]# ./redis-cli -p 7000 -c dbsize (integer) 369 [root@bogon bin]# ./redis-cli -p 7001 -c dbsize (integer) 316 [root@bogon bin]# ./redis-cli -p 7002 -c dbsize (integer) 316 # 发现基本上比较均匀1000个key平均分到三个分片上了
-
kill主节点
# 我们把7000的redis-server kill掉,看看是否能自动迁移到7003 [root@bogon bin]# ps aux | grep redis | grep 7000 root 11223 0.2 0.2 140836 9408 ? Ssl Aug23 0:47 ./redis-server *:7000 [cluster] [root@bogon bin]# kill -9 11223 [root@bogon bin]# ps aux | grep redis root 4212 0.2 0.2 140836 9196 ? Ssl Aug23 1:27 ./redis-server *:7003 [cluster] root 4214 0.2 0.1 140836 7412 ? Ssl Aug23 1:27 ./redis-server *:7001 [cluster] root 4218 0.2 0.1 140836 7352 ? Ssl Aug23 1:26 ./redis-server *:7002 [cluster] root 12649 0.2 0.2 140836 9552 ? Ssl Aug23 0:39 ./redis-server *:7004 [cluster]
-
查看当前节点状态
[root@bogon bin]# ./redis-cli -p 7001 -c cluster nodes f26de43ddafc2e8e000132c3b53473d42906b429 127.0.0.1:7004 slave 7e46925ca9781a403facf450aa0b4b8149ad9c27 0 1440403755252 2 connected 7e46925ca9781a403facf450aa0b4b8149ad9c27 127.0.0.1:7001 myself,master - 0 0 2 connected 5795-10922 36fce4bf6c829797c433c08b045a8c482e07fbc1 127.0.0.1:7000 master,fail - 1440403749785 1440403749179 5 disconnected 245d95199dcc1e01880b2876fcab8a75b1867b19 127.0.0.1:7003 master - 0 1440403756767 6 connected 0-5794 10923-11255 e89c01d7cc2a83f072f66d0c2fb1f5cd92c66888 127.0.0.1:7002 master - 0 1440403756260 3 connected 11256-16383 # 发现7003已经自动成为主了