centos8平台redis cluster集群搭建(redis5.0.7)
一，规划 redis cluster
1,cluster采用六台redis，3主3从
redis1 ： ip: 172.17.0.2
redis2 ： ip: 172.17.0.3
redis3 ： ip: 172.17.0.4
redis4 ： ip: 172.17.0.5
redis5 ： ip: 172.17.0.6
redis6 ： ip: 172.17.0.7
2,为什么需要6个节点？
redis主从集群最少需要6个节点
master节点至少要3个，slave节点也是3个，
因为一个redis集群要对外提供可用的服务，那么集群中必须要有过半的master节点正常工作。
所以如果想搭建一个能够允许 n 个master节点挂掉的集群，那么就要搭建2n+1个master节点的集群。
2个节点，一个宕掉，剩下的1个不超过1半(1)，集群停止工作
3个节点，一个宕掉，剩下的两个超过1半(1.5)，集群继续工作
4个节点，一个宕掉，剩下的3个超过1半（2个），集群继续工作
4个节点，两个宕掉，剩下的2个不超过1半（2个），集群停止工作
3个节点和4个节点，都是只允许一个节点宕掉，
可见3个是最实际的选择
说明：刘宏缔的架构森林是一个专注架构的博客，
网站：https://blog.imgtouch.com
本文: https://blog.imgtouch.com/index.php/2023/05/21/centos8linuxrediscluster-ji-qun-da-jian-redis507/
对应的源码可以访问这里获取： https://github.com/liuhongdi/
说明：作者:刘宏缔 邮箱: 371125307@qq.com
二，在每台机器上安装redis,操作相同
1，安装wget
[root@redis source]# yum install wget
2,安装gcc
[root@redis source]# yum install gcc
3,安装tcl
[root@redis source]# yum install tcl
4,安装make
[root@redis redis-5.0.7]# yum install make
5,下载redis
[root@redis source]# wget http://download.redis.io/releases/redis-5.0.7.tar.gz
6,解压redis
[root@redis source]# tar -zxvf redis-5.0.7.tar.gz
7,编译:
[root@redis source]# cd redis-5.0.7 [root@redis redis-5.0.7]# make MALLOC=libc
8,测试编译效果:
[root@redis redis-5.0.7]# make test
遇到报错：
You need tcl 8.5 or newer in order to run the Redis test
解决：我们已安装了tcl
make test找不到它是因为缺少which命令，
[root@redis redis-5.0.7]# yum install which
9，再来一次:
[root@redis redis-5.0.7]# make clean [root@redis redis-5.0.7]# make MALLOC=libc [root@redis redis-5.0.7]# make test
最终看到以下提示则表示编译无问题
\o/ All tests passed without errors!
10,安装
生成目录
[root@redis soft]# mkdir /usr/local/soft/redis5 [root@redis soft]# mkdir /usr/local/soft/redis5/bin [root@redis soft]# mkdir /usr/local/soft/redis5/conf
复制文件
[root@redis soft]# cp /usr/local/source/redis-5.0.7/src/redis-cli /usr/local/soft/redis5/bin/ [root@redis soft]# cp /usr/local/source/redis-5.0.7/src/redis-server /usr/local/soft/redis5/bin/ [root@redis soft]# cp /usr/local/source/redis-5.0.7/redis.conf /usr/local/soft/redis5/conf/
三，在每台机器上配置redis,注意有区别的地方:
1，生成目录:
[root@redis soft]# mkdir /data/ [root@redis soft]# mkdir /data/redis [root@redis soft]# mkdir /data/redis/data [root@redis soft]# mkdir /data/redis/log [root@redis soft]# mkdir /data/redis/cluster
说明：data存放rdb或aof
log存放日志
cluster存放cluster的配置文件nodes.conf
2,配置redis.conf
配置以下各项
#数据文件存储目录 dir /data/redis/data/ #日志级别 loglevel notice(使用这个默认值，不用变) #日志文件 logfile "/data/redis/log/redis.log" #是否以服务方式运行 daemonize yes #pid文件 pidfile /var/run/redis_6379.pid(使用这个默认值，不用变) #是否以cluster方式运行 cluster-enabled yes(此行取消注释即可) #cluster的配置缓存文件 cluster-config-file /data/redis/cluster/nodes-6379.conf #连接node的超时时间 cluster-node-timeout 15000(此行取消注释即可) #访问每台redis的密码 requirepass lhd123 #slave访问master的密码,注意与上一个相同 masterauth lhd123 #是否启用保护模式 protected-mode no #绑定ip:注意要改成自己当前机器的可访问的ip #如果是127.0.0.1,则只能从本机访问 bind 172.17.0.2
说明：这里要注意的一点：
绑定ip:注意要改成自己当前机器的可访问的ip
四，给每台机器上的redis生成systemd启动文件,操作相同
1,生成service文件
[root@redis conf]# vi /lib/systemd/system/redis.service
内容:
[Unit] Description=Redis After=network.target [Service] Type=forking PIDFile=/var/run/redis_6379.pid ExecStart=/usr/local/soft/redis5/bin/redis-server /usr/local/soft/redis5/conf/redis.conf ExecReload=/bin/kill -s HUP $MAINPID ExecStop=/bin/kill -s QUIT $MAINPID PrivateTmp=true [Install] WantedBy=multi-user.target
重载systemd服务
[root@redis conf]# systemctl daemon-reload
启动redis服务
[root@redis conf]# systemctl start redis
测试：
[root@redis conf]# /usr/local/soft/redis5/bin/redis-cli 127.0.0.1:6379> auth lhd123 OK 127.0.0.1:6379> keys * (empty list or set) 127.0.0.1:6379> set a aaa (error) CLUSTERDOWN Hash slot not served
系统提示hash slot还没分配,因为cluster还未创建
五，创建cluster,测试写入,(在任一台机器上操作)
1，创建cluster,在上面创建的任一台机器上操作即可
命令:
[root@redis1 /]# /usr/local/soft/redis5/bin/redis-cli -a lhd123 --cluster create 172.17.0.2:6379 172.17.0.3:6379 172.17.0.4:6379 172.17.0.5:6379 172.17.0.6:6379 172.17.0.7:6379 --cluster-replicas 1
在需要回答下面问题时：输入yes
Can I set the above configuration? (type 'yes' to accept): yes
看一个例子：
#-a: auth 访问密码
#--cluster create :用来创建集群
#--cluster-replicas :为集群中的每个主节点创建一个从节点
[root@redis1 conf]# /usr/local/soft/redis5/bin/redis-cli -a lhd123 --cluster create 172.17.0.2:6379 172.17.0.3:6379 172.17.0.4:6379 172.17.0.5:6379 172.17.0.6:6379 172.17.0.7:6379 --cluster-replicas 1 Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. >>> Performing hash slots allocation on 6 nodes... Master[0] -> Slots 0 - 5460 Master[1] -> Slots 5461 - 10922 Master[2] -> Slots 10923 - 16383 Adding replica 172.17.0.6:6379 to 172.17.0.2:6379 Adding replica 172.17.0.7:6379 to 172.17.0.3:6379 Adding replica 172.17.0.5:6379 to 172.17.0.4:6379 M: eb701616b26b5a350605ae3ea5f80e4fc79d84c3 172.17.0.2:6379 slots:[0-5460] (5461 slots) master M: 5bdaafe57b1c46f61c5910d3822633a516feb4ae 172.17.0.3:6379 slots:[5461-10922] (5462 slots) master M: e024e898a21f3f4051abfb0957046dc4a81ef947 172.17.0.4:6379 slots:[10923-16383] (5461 slots) master S: 1ca00d6a680fc1b0d617a46996eaaefc3636fd5a 172.17.0.5:6379 replicates e024e898a21f3f4051abfb0957046dc4a81ef947 S: 9cd94c491211542dbfae96002489c9b63a5a54e7 172.17.0.6:6379 replicates eb701616b26b5a350605ae3ea5f80e4fc79d84c3 S: 6f90338cef5af2aa9f0580cd660c80d2ea5fab82 172.17.0.7:6379 replicates 5bdaafe57b1c46f61c5910d3822633a516feb4ae Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join ... >>> Performing Cluster Check (using node 172.17.0.2:6379) M: eb701616b26b5a350605ae3ea5f80e4fc79d84c3 172.17.0.2:6379 slots:[0-5460] (5461 slots) master 1 additional replica(s) S: 1ca00d6a680fc1b0d617a46996eaaefc3636fd5a 172.17.0.5:6379 slots: (0 slots) slave replicates e024e898a21f3f4051abfb0957046dc4a81ef947 M: 5bdaafe57b1c46f61c5910d3822633a516feb4ae 172.17.0.3:6379 slots:[5461-10922] (5462 slots) master 1 additional replica(s) M: e024e898a21f3f4051abfb0957046dc4a81ef947 172.17.0.4:6379 slots:[10923-16383] (5461 slots) master 1 additional replica(s) S: 9cd94c491211542dbfae96002489c9b63a5a54e7 172.17.0.6:6379 slots: (0 slots) slave replicates eb701616b26b5a350605ae3ea5f80e4fc79d84c3 S: 6f90338cef5af2aa9f0580cd660c80d2ea5fab82 172.17.0.7:6379 slots: (0 slots) slave replicates 5bdaafe57b1c46f61c5910d3822633a516feb4ae [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
2,测试写入:
# -c: Enable cluster mode,用集群模式访问
# -h: Server hostname (default: 127.0.0.1),指定要访问的主机
[root@redis1 conf]# /usr/local/soft/redis5/bin/redis-cli -a lhd123 -c -h 172.17.0.2 Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 172.17.0.2:6379> set c bb -> Redirected to slot [7365] located at 172.17.0.3:6379 OK 172.17.0.3:6379> set d bb -> Redirected to slot [11298] located at 172.17.0.4:6379 OK 172.17.0.4:6379> get b -> Redirected to slot [3300] located at 172.17.0.2:6379 "bb" 172.17.0.2:6379> get c -> Redirected to slot [7365] located at 172.17.0.3:6379 "bb" 172.17.0.3:6379> get d -> Redirected to slot [11298] located at 172.17.0.4:6379 "bb"
六，查看cluster信息(在任一台机器上操作)
1，查看节点列表：
以集群方式登录redis-cli后查看
172.17.0.3:6379> CLUSTER NODES 9cd94c491211542dbfae96002489c9b63a5a54e7 172.17.0.6:6379@16379 slave eb701616b26b5a350605ae3ea5f80e4fc79d84c3 0 1586861138000 1 connected 5bdaafe57b1c46f61c5910d3822633a516feb4ae 172.17.0.3:6379@16379 myself,master - 0 1586861135000 2 connected 5461-10922 6f90338cef5af2aa9f0580cd660c80d2ea5fab82 172.17.0.7:6379@16379 slave 5bdaafe57b1c46f61c5910d3822633a516feb4ae 0 1586861137000 6 connected e024e898a21f3f4051abfb0957046dc4a81ef947 172.17.0.4:6379@16379 master - 0 1586861136668 3 connected 10923-16383 1ca00d6a680fc1b0d617a46996eaaefc3636fd5a 172.17.0.5:6379@16379 slave e024e898a21f3f4051abfb0957046dc4a81ef947 0 1586861138674 3 connected eb701616b26b5a350605ae3ea5f80e4fc79d84c3 172.17.0.2:6379@16379 master - 0 1586861136000 1 connected 0-5460
用 check命令查看
#--cluster check: 列出群集中的机器信息
[root@redis2 /]# /usr/local/soft/redis5/bin/redis-cli --cluster check -a lhd123 172.17.0.3:6379 Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 172.17.0.3:6379 (5bdaafe5...) -> 2 keys | 5462 slots | 1 slaves. 172.17.0.4:6379 (e024e898...) -> 3 keys | 5461 slots | 1 slaves. 172.17.0.2:6379 (eb701616...) -> 2 keys | 5461 slots | 1 slaves. [OK] 7 keys in 3 masters. 0.00 keys per slot on average. >>> Performing Cluster Check (using node 172.17.0.3:6379) M: 5bdaafe57b1c46f61c5910d3822633a516feb4ae 172.17.0.3:6379 slots:[5461-10922] (5462 slots) master 1 additional replica(s) S: 9cd94c491211542dbfae96002489c9b63a5a54e7 172.17.0.6:6379 slots: (0 slots) slave replicates eb701616b26b5a350605ae3ea5f80e4fc79d84c3 S: 6f90338cef5af2aa9f0580cd660c80d2ea5fab82 172.17.0.7:6379 slots: (0 slots) slave replicates 5bdaafe57b1c46f61c5910d3822633a516feb4ae M: e024e898a21f3f4051abfb0957046dc4a81ef947 172.17.0.4:6379 slots:[10923-16383] (5461 slots) master 1 additional replica(s) S: 1ca00d6a680fc1b0d617a46996eaaefc3636fd5a 172.17.0.5:6379 slots: (0 slots) slave replicates e024e898a21f3f4051abfb0957046dc4a81ef947 M: eb701616b26b5a350605ae3ea5f80e4fc79d84c3 172.17.0.2:6379 slots:[0-5460] (5461 slots) master 1 additional replica(s) [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
2,查看节点的信息:
使用CLUSTER INFO命令
#CLUSTER INFO:cluster的统计信息
172.17.0.3:6379> CLUSTER INFO cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:6 cluster_my_epoch:2 cluster_stats_messages_ping_sent:2071 cluster_stats_messages_pong_sent:1919 cluster_stats_messages_meet_sent:1 cluster_stats_messages_sent:3991 cluster_stats_messages_ping_received:1915 cluster_stats_messages_pong_received:2072 cluster_stats_messages_meet_received:4 cluster_stats_messages_received:3991
七，创建redis cluster时遇到的报错或问题：
1,创建cluster时，停在Waiting for the cluster to join不动了
1，16379这个集群管理端口被防火墙做了拦截
2, docker环境中,nodes-6379.conf 文件中redis节点id如果相同也会有这个问题
删除文件后重启
例:
[root@redis1 conf]# rm /data/redis/cluster/nodes-6379.conf rm: remove regular file '/data/redis/cluster/nodes-6379.conf'? y
2,客户端写入时报错：
(error) MOVED
例子：
172.17.0.2:6379> set a aaa (error) MOVED 15495 172.17.0.4:6379
解决：
因为启动redis-cli时没有设置集群模式所导致
[root@redis1 conf]# /usr/local/soft/redis5/bin/redis-cli -c -h 172.17.0.2
3，客户端写入时报错：
(error) NOAUTH Authentication required.
例子：
[root@redis1 conf]# /usr/local/soft/redis5/bin/redis-cli -c -h 172.17.0.2 172.17.0.2:6379> auth lhd123 OK 172.17.0.2:6379> set a aaaa -> Redirected to slot [15495] located at 172.17.0.4:6379 (error) NOAUTH Authentication required. 172.17.0.4:6379> auth "lhd123" OK 172.17.0.4:6379> set a aaaa OK
解决：
[root@redis1 conf]# /usr/local/soft/redis5/bin/redis-cli -a lhd123 -c -h 172.17.0.2
八，查看redis版本
[root@redis1 /]# /usr/local/soft/redis5/bin/redis-server --version Redis server v=5.0.7 sha=00000000:0 malloc=libc bits=64 build=c52ab39fadfc446c
九，查看centos版本
[root@redis1 /]# cat /etc/redhat-release CentOS Linux release 8.1.1911 (Core)
